2024-04-27T06:49:43.941 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-27T06:49:43.941 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi --verbose --archive /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037 --owner scheduled_teuthology@teuthology --description fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} -- /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037/orig.config.yaml 2024-04-27T06:49:44.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T06:49:44.163 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037 branch: main description: fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672037' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 seed: 4710 sha1: c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f sleep_before_teardown: 0 subset: 356/512 suite: fs suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e213a58e787437ff508e6c890ccd5fd32dc96b5 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBByKoS5wtV294iofbbkD3c3JmT9A/H+k//+OspIbHvtlCiRYHBcAtpsx8ZGlqxT7/kBjYs8ihx6uCGZlvOLDaJU= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMRr6fEgH9O7yAPsE7IESr/UoTs+hNtQFq2EUtgh9JWnpxRplmFRVRRPDppla624f4eBnmwhreWF1I7Z9teX5fk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cephfs_shell teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_20:24:13 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-27T06:49:44.163 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa; will attempt to use it 2024-04-27T06:49:44.163 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa/tasks 2024-04-27T06:49:44.164 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-27T06:49:44.229 INFO:teuthology.task.internal:Checking packages... 2024-04-27T06:49:44.263 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f' 2024-04-27T06:49:44.263 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T06:49:44.263 INFO:teuthology.packaging:ref: None 2024-04-27T06:49:44.263 INFO:teuthology.packaging:tag: None 2024-04-27T06:49:44.263 INFO:teuthology.packaging:branch: main 2024-04-27T06:49:44.263 INFO:teuthology.packaging:sha1: c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-27T06:49:44.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-04-27T06:49:44.508 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3253-g4e213a58-1jammy 2024-04-27T06:49:44.510 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-27T06:49:44.570 INFO:teuthology.task.internal:no buildpackages task found 2024-04-27T06:49:44.570 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-27T06:49:44.595 INFO:teuthology.task.internal:Saving configuration 2024-04-27T06:49:44.611 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-27T06:49:44.631 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T06:49:44.653 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 06:44:27.756996', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEKfvQdxaEsY84W+3YUBbF1z/F1NjQfnOzQUEhsQKhva'} 2024-04-27T06:49:44.673 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 06:44:27.755952', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMVq8ztWcA+Ijg5YrjKieoIHAJgds71fHK0wt7ZVQ6SO'} 2024-04-27T06:49:44.673 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-27T06:49:44.680 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-27T06:49:44.680 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-27T06:49:44.680 INFO:teuthology.run_tasks:Running task console_log... 2024-04-27T06:49:44.747 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9b904733a0>, signals=[15]) 2024-04-27T06:49:44.747 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-27T06:49:44.754 INFO:teuthology.task.internal:Opening connections... 2024-04-27T06:49:44.754 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T06:49:44.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T06:49:44.844 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T06:49:44.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T06:49:44.923 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-27T06:49:44.930 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-04-27T06:49:44.936 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-04-27T06:49:44.936 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-04-27T06:49:44.982 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-27T06:49:44.982 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu" 2024-04-27T06:49:44.982 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="22.04" 2024-04-27T06:49:44.982 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-27T06:49:44.983 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=jammy 2024-04-27T06:49:44.983 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu 2024-04-27T06:49:44.983 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian 2024-04-27T06:49:44.983 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-27T06:49:44.983 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-27T06:49:44.983 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-27T06:49:44.983 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-27T06:49:44.983 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=jammy 2024-04-27T06:49:44.984 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-04-27T06:49:45.002 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-04-27T06:49:45.008 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-04-27T06:49:45.008 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-04-27T06:49:45.055 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-27T06:49:45.055 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Ubuntu" 2024-04-27T06:49:45.056 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="22.04" 2024-04-27T06:49:45.056 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-27T06:49:45.056 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_CODENAME=jammy 2024-04-27T06:49:45.056 INFO:teuthology.orchestra.run.smithi053.stdout:ID=ubuntu 2024-04-27T06:49:45.056 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE=debian 2024-04-27T06:49:45.056 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-27T06:49:45.056 INFO:teuthology.orchestra.run.smithi053.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-27T06:49:45.056 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-27T06:49:45.056 INFO:teuthology.orchestra.run.smithi053.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-27T06:49:45.056 INFO:teuthology.orchestra.run.smithi053.stdout:UBUNTU_CODENAME=jammy 2024-04-27T06:49:45.057 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-04-27T06:49:45.076 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-27T06:49:45.086 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-27T06:49:45.092 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-27T06:49:45.092 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-04-27T06:49:45.095 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-04-27T06:49:45.103 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-27T06:49:45.109 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-27T06:49:45.109 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-04-27T06:49:45.139 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-04-27T06:49:45.154 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-27T06:49:45.201 INFO:teuthology.run_tasks:Running task kernel... 2024-04-27T06:49:45.216 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-27T06:49:45.216 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-27T06:49:45.217 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T06:49:45.217 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T06:49:45.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T06:49:45.222 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-04-27T06:49:45.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T06:49:45.223 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-04-27T06:49:45.268 INFO:teuthology.orchestra.run.smithi053.stdout:5.15.0-56-generic 2024-04-27T06:49:45.268 INFO:teuthology.task.kernel:Running kernel on smithi053: 5.15.0-56-generic 2024-04-27T06:49:45.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-get clean 2024-04-27T06:49:45.270 INFO:teuthology.orchestra.run.smithi012.stdout:5.15.0-56-generic 2024-04-27T06:49:45.271 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.15.0-56-generic 2024-04-27T06:49:45.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2024-04-27T06:49:45.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-get update 2024-04-27T06:49:45.404 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-04-27T06:49:45.532 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T06:49:45.533 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-27T06:49:45.533 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-27T06:49:45.566 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T06:49:45.568 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-27T06:49:45.568 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-27T06:49:45.621 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-27T06:49:45.648 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-27T06:49:45.772 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-27T06:49:45.804 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-27T06:49:45.857 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-27T06:49:45.858 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-27T06:49:45.894 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-27T06:49:45.894 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-27T06:49:45.902 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-27T06:49:45.908 INFO:teuthology.orchestra.run.smithi053.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-27T06:49:45.908 INFO:teuthology.orchestra.run.smithi053.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-27T06:49:45.930 INFO:teuthology.orchestra.run.smithi053.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-27T06:49:45.931 INFO:teuthology.orchestra.run.smithi053.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-27T06:49:45.934 INFO:teuthology.orchestra.run.smithi053.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-27T06:49:45.934 INFO:teuthology.orchestra.run.smithi053.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-27T06:49:45.937 INFO:teuthology.orchestra.run.smithi053.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-27T06:49:45.942 INFO:teuthology.orchestra.run.smithi053.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-27T06:49:45.948 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-27T06:49:45.955 INFO:teuthology.orchestra.run.smithi053.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-27T06:49:45.956 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-27T06:49:45.956 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-27T06:49:45.957 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-27T06:49:45.958 INFO:teuthology.orchestra.run.smithi053.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-27T06:49:45.968 INFO:teuthology.orchestra.run.smithi053.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-27T06:49:45.985 INFO:teuthology.orchestra.run.smithi053.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-27T06:49:45.988 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-27T06:49:45.993 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-27T06:49:45.993 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-27T06:49:46.001 INFO:teuthology.orchestra.run.smithi053.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-27T06:49:46.002 INFO:teuthology.orchestra.run.smithi053.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-27T06:49:46.002 INFO:teuthology.orchestra.run.smithi053.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-27T06:49:46.005 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-27T06:49:46.013 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-27T06:49:46.014 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-27T06:49:46.015 INFO:teuthology.orchestra.run.smithi053.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-27T06:49:46.017 INFO:teuthology.orchestra.run.smithi053.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-27T06:49:46.017 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-27T06:49:46.018 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-27T06:49:46.018 INFO:teuthology.orchestra.run.smithi053.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-27T06:49:46.041 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-27T06:49:46.041 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-27T06:49:46.041 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-27T06:49:46.045 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-27T06:49:46.051 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-27T06:49:46.051 INFO:teuthology.orchestra.run.smithi053.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-27T06:49:46.052 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-27T06:49:46.052 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-27T06:49:46.056 INFO:teuthology.orchestra.run.smithi053.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-27T06:49:46.057 INFO:teuthology.orchestra.run.smithi053.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-27T06:49:46.066 INFO:teuthology.orchestra.run.smithi053.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,071 kB] 2024-04-27T06:49:46.076 INFO:teuthology.orchestra.run.smithi053.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-27T06:49:46.079 INFO:teuthology.orchestra.run.smithi053.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-27T06:49:46.079 INFO:teuthology.orchestra.run.smithi053.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-27T06:49:46.080 INFO:teuthology.orchestra.run.smithi053.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-27T06:49:46.080 INFO:teuthology.orchestra.run.smithi053.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-27T06:49:46.081 INFO:teuthology.orchestra.run.smithi053.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-27T06:49:46.081 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-27T06:49:46.085 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-27T06:49:46.086 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,071 kB] 2024-04-27T06:49:46.097 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-27T06:49:46.104 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-27T06:49:46.106 INFO:teuthology.orchestra.run.smithi053.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-27T06:49:46.107 INFO:teuthology.orchestra.run.smithi053.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-27T06:49:46.108 INFO:teuthology.orchestra.run.smithi053.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-27T06:49:46.108 INFO:teuthology.orchestra.run.smithi053.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-27T06:49:46.108 INFO:teuthology.orchestra.run.smithi053.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-27T06:49:46.108 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-27T06:49:46.108 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-27T06:49:46.108 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-27T06:49:46.109 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-27T06:49:46.109 INFO:teuthology.orchestra.run.smithi053.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-27T06:49:46.109 INFO:teuthology.orchestra.run.smithi053.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-27T06:49:46.109 INFO:teuthology.orchestra.run.smithi053.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-27T06:49:46.116 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-27T06:49:46.116 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-27T06:49:46.117 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-27T06:49:46.128 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-27T06:49:46.129 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-27T06:49:46.129 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-27T06:49:46.129 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-27T06:49:46.129 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-27T06:49:46.135 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-27T06:49:51.926 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 13.2 MB in 2s (6,915 kB/s) 2024-04-27T06:49:52.096 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 13.2 MB in 2s (6,840 kB/s) 2024-04-27T06:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-04-27T06:49:53.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-27T06:49:53.379 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-04-27T06:49:53.496 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-27T06:49:53.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-27T06:49:53.579 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-27T06:49:53.582 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2024-04-27T06:49:53.583 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2024-04-27T06:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T06:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T06:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T06:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T06:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-27T06:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: python2.7-minimal 2024-04-27T06:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T06:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout:The following additional packages will be installed: 2024-04-27T06:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-27T06:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-27T06:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-27T06:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout:Suggested packages: 2024-04-27T06:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-27T06:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout:Recommended packages: 2024-04-27T06:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: thermald 2024-04-27T06:49:53.783 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-04-27T06:49:53.784 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-04-27T06:49:53.829 INFO:teuthology.orchestra.run.smithi053.stdout:The following NEW packages will be installed: 2024-04-27T06:49:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-27T06:49:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-27T06:49:53.829 INFO:teuthology.orchestra.run.smithi053.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-27T06:49:53.830 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages will be upgraded: 2024-04-27T06:49:53.831 INFO:teuthology.orchestra.run.smithi053.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-27T06:49:53.966 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T06:49:53.966 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T06:49:53.966 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T06:49:53.967 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T06:49:53.967 INFO:teuthology.orchestra.run.smithi012.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-27T06:49:53.967 INFO:teuthology.orchestra.run.smithi012.stdout: python2.7-minimal 2024-04-27T06:49:53.967 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T06:49:53.970 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-04-27T06:49:53.970 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-27T06:49:53.970 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-27T06:49:53.970 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-27T06:49:53.970 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-04-27T06:49:53.970 INFO:teuthology.orchestra.run.smithi012.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-27T06:49:53.971 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2024-04-27T06:49:53.971 INFO:teuthology.orchestra.run.smithi012.stdout: thermald 2024-04-27T06:49:54.011 INFO:teuthology.orchestra.run.smithi053.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-27T06:49:54.011 INFO:teuthology.orchestra.run.smithi053.stdout:Need to get 113 MB of archives. 2024-04-27T06:49:54.011 INFO:teuthology.orchestra.run.smithi053.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-27T06:49:54.011 INFO:teuthology.orchestra.run.smithi053.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-27T06:49:54.022 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-04-27T06:49:54.022 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-27T06:49:54.022 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-27T06:49:54.023 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-27T06:49:54.024 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2024-04-27T06:49:54.025 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-27T06:49:54.082 INFO:teuthology.orchestra.run.smithi012.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-27T06:49:54.083 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 113 MB of archives. 2024-04-27T06:49:54.083 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-27T06:49:54.083 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-27T06:49:54.563 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-27T06:49:54.691 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-27T06:49:55.440 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-27T06:49:55.695 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-27T06:49:55.696 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-27T06:49:55.696 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-27T06:49:55.787 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-27T06:49:55.997 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-27T06:49:56.034 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-27T06:49:56.452 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 113 MB in 2s (56.6 MB/s) 2024-04-27T06:49:56.642 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-27T06:49:58.275 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-27T06:49:58.275 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-27T06:49:58.276 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-27T06:49:58.796 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-27T06:49:58.903 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-27T06:49:59.004 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-27T06:49:59.008 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T06:49:59.223 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:49:59.305 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 113 MB in 5s (22.4 MB/s) 2024-04-27T06:49:59.474 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-27T06:50:01.391 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-27T06:50:01.413 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T06:50:01.509 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:01.511 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-27T06:50:01.516 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T06:50:01.589 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:01.994 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-27T06:50:02.016 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T06:50:02.061 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:03.792 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-27T06:50:03.813 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T06:50:03.900 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:04.344 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-27T06:50:04.365 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T06:50:04.395 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:08.753 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T06:50:08.880 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T06:50:09.214 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T06:50:09.342 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T06:50:09.661 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-27T06:50:09.685 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-27T06:50:09.719 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-27T06:50:10.640 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T06:50:10.750 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T06:50:11.083 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T06:50:11.204 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T06:50:11.523 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-27T06:50:11.546 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-27T06:50:11.582 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-27T06:50:15.124 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-27T06:50:15.161 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T06:50:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:17.238 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-27T06:50:17.257 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-27T06:50:17.288 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:17.822 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T06:50:17.927 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T06:50:18.261 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-27T06:50:18.382 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:18.508 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-27T06:50:18.611 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:19.937 INFO:teuthology.orchestra.run.smithi012.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-27T06:50:19.937 INFO:teuthology.orchestra.run.smithi012.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-27T06:50:19.937 INFO:teuthology.orchestra.run.smithi012.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-27T06:50:19.937 INFO:teuthology.orchestra.run.smithi012.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-27T06:50:19.973 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-27T06:50:20.088 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:20.198 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-27T06:50:20.497 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-27T06:50:20.602 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:20.720 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-27T06:50:20.848 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:21.966 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:22.211 INFO:teuthology.orchestra.run.smithi053.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-27T06:50:22.211 INFO:teuthology.orchestra.run.smithi053.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-27T06:50:22.212 INFO:teuthology.orchestra.run.smithi053.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-27T06:50:22.212 INFO:teuthology.orchestra.run.smithi053.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-27T06:50:22.376 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:23.812 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:23.816 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-27T06:50:23.917 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-27T06:50:24.026 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:24.080 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-27T06:50:24.080 INFO:teuthology.orchestra.run.smithi012.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-27T06:50:25.263 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-27T06:50:25.388 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-27T06:50:25.506 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-27T06:50:25.566 INFO:teuthology.orchestra.run.smithi053.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-27T06:50:25.566 INFO:teuthology.orchestra.run.smithi053.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-27T06:50:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-27T06:50:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub' 2024-04-27T06:50:38.333 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-27T06:50:38.357 INFO:teuthology.orchestra.run.smithi012.stdout:Generating grub configuration file ... 2024-04-27T06:50:38.618 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-27T06:50:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-27T06:50:38.933 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-27T06:50:38.939 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-27T06:50:39.020 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-27T06:50:39.025 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-27T06:50:39.183 INFO:teuthology.orchestra.run.smithi012.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-27T06:50:39.183 INFO:teuthology.orchestra.run.smithi012.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-27T06:50:39.183 INFO:teuthology.orchestra.run.smithi012.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-27T06:50:39.226 INFO:teuthology.orchestra.run.smithi012.stdout:done 2024-04-27T06:50:39.719 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2024-04-27T06:50:39.738 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2024-04-27T06:50:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2024-04-27T06:50:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2024-04-27T06:50:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2024-04-27T06:50:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 21 2024-04-27T06:50:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2024-04-27T06:50:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2024-04-27T06:50:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2024-04-27T06:50:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.15.0.105.102 2024-04-27T06:50:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-27T06:50:39.739 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-27T06:50:39.740 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2024-04-27T06:50:39.740 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2024-04-27T06:50:39.740 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2024-04-27T06:50:39.740 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2024-04-27T06:50:39.740 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-27T06:50:39.740 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-27T06:50:39.741 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-27T06:50:39.741 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-27T06:50:39.741 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2024-04-27T06:50:39.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2024-04-27T06:50:39.910 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-04-27T06:50:40.051 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-27T06:50:40.051 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T06:50:40.065 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-27T06:50:40.080 INFO:teuthology.orchestra.run.smithi053.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-27T06:50:40.080 INFO:teuthology.orchestra.run.smithi053.stdout:Sourcing file `/etc/default/grub' 2024-04-27T06:50:40.085 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-27T06:50:40.103 INFO:teuthology.orchestra.run.smithi053.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-27T06:50:40.129 INFO:teuthology.orchestra.run.smithi053.stdout:Generating grub configuration file ... 2024-04-27T06:50:40.760 INFO:teuthology.orchestra.run.smithi053.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-27T06:50:40.778 INFO:teuthology.orchestra.run.smithi053.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-27T06:50:41.126 INFO:teuthology.orchestra.run.smithi053.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-27T06:50:41.131 INFO:teuthology.orchestra.run.smithi053.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-27T06:50:41.195 INFO:teuthology.orchestra.run.smithi053.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-27T06:50:41.201 INFO:teuthology.orchestra.run.smithi053.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-27T06:50:41.349 INFO:teuthology.orchestra.run.smithi053.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-27T06:50:41.349 INFO:teuthology.orchestra.run.smithi053.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-27T06:50:41.349 INFO:teuthology.orchestra.run.smithi053.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-27T06:50:41.394 INFO:teuthology.orchestra.run.smithi053.stdout:done 2024-04-27T06:50:41.461 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-27T06:50:41.479 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-27T06:50:41.541 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-27T06:50:41.745 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-04-27T06:50:41.746 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-04-27T06:50:41.929 INFO:teuthology.orchestra.run.smithi012.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-27T06:50:41.929 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T06:50:41.930 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T06:50:41.930 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T06:50:41.930 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T06:50:41.930 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T06:50:41.930 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T06:50:41.930 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T06:50:41.937 DEBUG:teuthology.orchestra.run.smithi053:> dpkg -s linux-image-generic 2024-04-27T06:50:41.960 INFO:teuthology.orchestra.run.smithi053.stdout:Package: linux-image-generic 2024-04-27T06:50:41.960 INFO:teuthology.orchestra.run.smithi053.stdout:Status: install ok installed 2024-04-27T06:50:41.960 INFO:teuthology.orchestra.run.smithi053.stdout:Priority: optional 2024-04-27T06:50:41.960 INFO:teuthology.orchestra.run.smithi053.stdout:Section: kernel 2024-04-27T06:50:41.960 INFO:teuthology.orchestra.run.smithi053.stdout:Installed-Size: 21 2024-04-27T06:50:41.960 INFO:teuthology.orchestra.run.smithi053.stdout:Maintainer: Ubuntu Kernel Team 2024-04-27T06:50:41.961 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: amd64 2024-04-27T06:50:41.961 INFO:teuthology.orchestra.run.smithi053.stdout:Source: linux-meta 2024-04-27T06:50:41.961 INFO:teuthology.orchestra.run.smithi053.stdout:Version: 5.15.0.105.102 2024-04-27T06:50:41.961 INFO:teuthology.orchestra.run.smithi053.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-27T06:50:41.961 INFO:teuthology.orchestra.run.smithi053.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-27T06:50:41.961 INFO:teuthology.orchestra.run.smithi053.stdout:Recommends: thermald 2024-04-27T06:50:41.961 INFO:teuthology.orchestra.run.smithi053.stdout:Description: Generic Linux kernel image 2024-04-27T06:50:41.961 INFO:teuthology.orchestra.run.smithi053.stdout: This package will always depend on the latest generic kernel image 2024-04-27T06:50:41.961 INFO:teuthology.orchestra.run.smithi053.stdout: available. 2024-04-27T06:50:41.961 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-27T06:50:41.961 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-27T06:50:41.961 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-27T06:50:41.961 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-27T06:50:41.962 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=None, version=distro) 2024-04-27T06:50:41.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-get clean 2024-04-27T06:50:41.975 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-27T06:50:41.977 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2024-04-27T06:50:41.993 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 21 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.15.0.105.102 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2024-04-27T06:50:41.994 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2024-04-27T06:50:41.995 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-04-27T06:50:42.038 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.eNKOwftt5o 2024-04-27T06:50:42.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub/grub.cfg /tmp/tmp.eNKOwftt5o 2024-04-27T06:50:42.074 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-get update 2024-04-27T06:50:42.094 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.eNKOwftt5o 2024-04-27T06:50:42.215 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.eNKOwftt5o is 10KB 2024-04-27T06:50:42.229 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.eNKOwftt5o 2024-04-27T06:50:42.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-27T06:50:42.292 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T06:50:42.292 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-27T06:50:42.292 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-27T06:50:42.309 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-27T06:50:42.310 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T06:50:42.364 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-27T06:50:42.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo update-grub 2024-04-27T06:50:42.390 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-27T06:50:42.472 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-27T06:50:43.529 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2024-04-27T06:50:43.530 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-27T06:50:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-04-27T06:50:43.680 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-04-27T06:50:43.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-27T06:50:43.703 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-27T06:50:43.709 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-27T06:50:43.764 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-04-27T06:50:43.938 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-27T06:50:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-27T06:50:43.970 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2024-04-27T06:50:43.971 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2024-04-27T06:50:43.990 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-27T06:50:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-27T06:50:44.129 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-27T06:50:44.129 INFO:teuthology.orchestra.run.smithi012.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-27T06:50:44.129 INFO:teuthology.orchestra.run.smithi012.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-27T06:50:44.145 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-04-27T06:50:44.147 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-04-27T06:50:44.157 INFO:teuthology.orchestra.run.smithi053.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-27T06:50:44.157 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T06:50:44.157 INFO:teuthology.orchestra.run.smithi053.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T06:50:44.157 INFO:teuthology.orchestra.run.smithi053.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T06:50:44.157 INFO:teuthology.orchestra.run.smithi053.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T06:50:44.158 INFO:teuthology.orchestra.run.smithi053.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T06:50:44.158 INFO:teuthology.orchestra.run.smithi053.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T06:50:44.158 INFO:teuthology.orchestra.run.smithi053.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T06:50:44.202 INFO:teuthology.orchestra.run.smithi053.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-27T06:50:44.204 DEBUG:teuthology.orchestra.run.smithi053:> dpkg -s linux-image-generic 2024-04-27T06:50:44.220 INFO:teuthology.orchestra.run.smithi053.stdout:Package: linux-image-generic 2024-04-27T06:50:44.220 INFO:teuthology.orchestra.run.smithi053.stdout:Status: install ok installed 2024-04-27T06:50:44.220 INFO:teuthology.orchestra.run.smithi053.stdout:Priority: optional 2024-04-27T06:50:44.220 INFO:teuthology.orchestra.run.smithi053.stdout:Section: kernel 2024-04-27T06:50:44.220 INFO:teuthology.orchestra.run.smithi053.stdout:Installed-Size: 21 2024-04-27T06:50:44.220 INFO:teuthology.orchestra.run.smithi053.stdout:Maintainer: Ubuntu Kernel Team 2024-04-27T06:50:44.221 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: amd64 2024-04-27T06:50:44.221 INFO:teuthology.orchestra.run.smithi053.stdout:Source: linux-meta 2024-04-27T06:50:44.221 INFO:teuthology.orchestra.run.smithi053.stdout:Version: 5.15.0.105.102 2024-04-27T06:50:44.221 INFO:teuthology.orchestra.run.smithi053.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-27T06:50:44.221 INFO:teuthology.orchestra.run.smithi053.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-27T06:50:44.221 INFO:teuthology.orchestra.run.smithi053.stdout:Recommends: thermald 2024-04-27T06:50:44.221 INFO:teuthology.orchestra.run.smithi053.stdout:Description: Generic Linux kernel image 2024-04-27T06:50:44.221 INFO:teuthology.orchestra.run.smithi053.stdout: This package will always depend on the latest generic kernel image 2024-04-27T06:50:44.221 INFO:teuthology.orchestra.run.smithi053.stdout: available. 2024-04-27T06:50:44.221 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-04-27T06:50:44.271 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.PFcMilcCeG 2024-04-27T06:50:44.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub/grub.cfg /tmp/tmp.PFcMilcCeG 2024-04-27T06:50:44.327 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.PFcMilcCeG 2024-04-27T06:50:44.437 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.PFcMilcCeG is 10KB 2024-04-27T06:50:44.450 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.PFcMilcCeG 2024-04-27T06:50:44.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-27T06:50:44.513 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T06:50:44.514 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-27T06:50:44.514 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-27T06:50:44.587 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-27T06:50:44.587 DEBUG:teuthology.orchestra.run.smithi053:> sudo update-grub 2024-04-27T06:50:45.688 INFO:teuthology.orchestra.run.smithi053.stderr:Sourcing file `/etc/default/grub' 2024-04-27T06:50:45.689 INFO:teuthology.orchestra.run.smithi053.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-27T06:50:45.692 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2024-04-27T06:50:45.883 INFO:teuthology.orchestra.run.smithi053.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-27T06:50:45.890 INFO:teuthology.orchestra.run.smithi053.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-27T06:50:46.113 INFO:teuthology.orchestra.run.smithi053.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-27T06:50:46.119 INFO:teuthology.orchestra.run.smithi053.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-27T06:50:46.162 INFO:teuthology.orchestra.run.smithi053.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-27T06:50:46.167 INFO:teuthology.orchestra.run.smithi053.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-27T06:50:46.302 INFO:teuthology.orchestra.run.smithi053.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-27T06:50:46.302 INFO:teuthology.orchestra.run.smithi053.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-27T06:50:46.302 INFO:teuthology.orchestra.run.smithi053.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-27T06:50:46.321 INFO:teuthology.orchestra.run.smithi053.stderr:done 2024-04-27T06:50:46.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2024-04-27T06:51:14.175 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-27T06:51:14.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-27T06:51:14.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T06:51:16.328 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-27T06:51:16.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-27T06:51:16.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T06:51:32.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-04-27T06:51:34.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-04-27T06:51:41.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-27T06:51:41.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T06:51:43.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-27T06:51:43.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T06:52:02.209 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-04-27T06:52:14.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-27T06:52:14.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T06:52:17.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.53 2024-04-27T06:52:32.291 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-27T06:52:32.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T06:52:32.517 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-04-27T06:52:32.975 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-27T06:52:32.975 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-27T06:52:32.975 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-04-27T06:52:33.020 INFO:teuthology.orchestra.run.smithi053.stdout:5.15.0-105-generic 2024-04-27T06:52:33.021 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-27T06:52:33.021 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-27T06:52:33.021 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-27T06:52:34.022 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-27T06:52:34.022 DEBUG:teuthology.orchestra.run.smithi053:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T06:52:34.107 INFO:teuthology.orchestra.run.smithi053.stdout:ttyS1 2024-04-27T06:52:34.138 DEBUG:teuthology.parallel:result is None 2024-04-27T06:52:41.522 DEBUG:teuthology.orchestra.remote:timed out 2024-04-27T06:52:53.523 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-27T06:52:53.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T06:52:53.742 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-04-27T06:52:54.282 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-27T06:52:54.283 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-27T06:52:54.283 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-04-27T06:52:54.330 INFO:teuthology.orchestra.run.smithi012.stdout:5.15.0-105-generic 2024-04-27T06:52:54.330 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-27T06:52:54.330 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-27T06:52:54.330 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-27T06:52:55.331 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-27T06:52:55.331 DEBUG:teuthology.orchestra.run.smithi012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T06:52:55.403 INFO:teuthology.orchestra.run.smithi012.stdout:ttyS1 2024-04-27T06:52:55.431 DEBUG:teuthology.parallel:result is None 2024-04-27T06:52:55.432 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-27T06:52:55.446 INFO:teuthology.task.internal:Creating test directory... 2024-04-27T06:52:55.446 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T06:52:55.448 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T06:52:55.454 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-27T06:52:55.464 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-27T06:52:55.478 INFO:teuthology.task.internal:Creating archive directory... 2024-04-27T06:52:55.479 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T06:52:55.495 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T06:52:55.519 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-27T06:52:55.530 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-27T06:52:55.530 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T06:52:55.547 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T06:52:55.562 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T06:52:55.570 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T06:52:55.582 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T06:52:55.590 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T06:52:55.592 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-27T06:52:55.603 INFO:teuthology.task.internal:Configuring sudo... 2024-04-27T06:52:55.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T06:52:55.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T06:52:55.649 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-27T06:52:55.661 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-27T06:52:55.661 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T06:52:55.672 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T06:52:55.697 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T06:52:55.723 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T06:52:55.771 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T06:52:55.771 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T06:52:55.834 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T06:52:55.840 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T06:52:55.889 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T06:52:55.889 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T06:52:55.953 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-04-27T06:52:55.956 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-04-27T06:52:56.031 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-27T06:52:56.040 INFO:teuthology.task.internal:Starting timer... 2024-04-27T06:52:56.040 INFO:teuthology.run_tasks:Running task pcp... 2024-04-27T06:52:56.062 INFO:teuthology.run_tasks:Running task selinux... 2024-04-27T06:52:56.076 DEBUG:teuthology.task.selinux:Excluding smithi012: OS 'ubuntu' does not support SELinux 2024-04-27T06:52:56.077 DEBUG:teuthology.task.selinux:Excluding smithi053: OS 'ubuntu' does not support SELinux 2024-04-27T06:52:56.077 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-27T06:52:56.077 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-27T06:52:56.077 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-27T06:52:56.077 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-27T06:52:56.089 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-27T06:52:56.106 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-27T06:52:56.108 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi053.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-27T07:00:07.241 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com')] 2024-04-27T07:00:07.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-27T07:00:07.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:00:07.329 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-04-27T07:00:07.406 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-27T07:00:07.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-27T07:00:07.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T07:00:07.491 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-04-27T07:00:07.570 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-04-27T07:00:07.571 INFO:teuthology.run_tasks:Running task clock... 2024-04-27T07:00:07.582 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-27T07:00:07.582 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T07:00:07.582 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T07:00:07.585 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T07:00:07.585 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T07:00:07.614 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-27T07:00:07.614 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: Command line: ntpd -gq 2024-04-27T07:00:07.614 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: ---------------------------------------------------- 2024-04-27T07:00:07.614 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: ntp-4 is maintained by Network Time Foundation, 2024-04-27T07:00:07.615 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-27T07:00:07.615 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: corporation. Support and training for ntp-4 are 2024-04-27T07:00:07.615 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: available at https://www.nwtime.org/support 2024-04-27T07:00:07.615 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: ---------------------------------------------------- 2024-04-27T07:00:07.615 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: proto: precision = 0.045 usec (-24) 2024-04-27T07:00:07.615 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: basedate set to 2022-02-04 2024-04-27T07:00:07.615 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: gps base set to 2022-02-06 (week 2196) 2024-04-27T07:00:07.615 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-27T07:00:07.615 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: restrict ::: KOD does nothing without LIMITED. 2024-04-27T07:00:07.616 INFO:teuthology.orchestra.run.smithi012.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-27T07:00:07.616 INFO:teuthology.orchestra.run.smithi012.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-27T07:00:07.617 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: Listen and drop on 0 v6wildcard [::]:123 2024-04-27T07:00:07.617 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-27T07:00:07.617 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: Listen normally on 2 lo 127.0.0.1:123 2024-04-27T07:00:07.617 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: Listen normally on 3 ens1f0 172.21.15.12:123 2024-04-27T07:00:07.617 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: Listen normally on 4 lo [::1]:123 2024-04-27T07:00:07.617 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25ea%4]:123 2024-04-27T07:00:07.617 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:07 ntpd[17133]: Listening on routing socket on fd #22 for interface updates 2024-04-27T07:00:07.641 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-27T07:00:07.641 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: Command line: ntpd -gq 2024-04-27T07:00:07.641 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: ---------------------------------------------------- 2024-04-27T07:00:07.642 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: ntp-4 is maintained by Network Time Foundation, 2024-04-27T07:00:07.642 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-27T07:00:07.642 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: corporation. Support and training for ntp-4 are 2024-04-27T07:00:07.642 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: available at https://www.nwtime.org/support 2024-04-27T07:00:07.642 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: ---------------------------------------------------- 2024-04-27T07:00:07.642 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: proto: precision = 0.066 usec (-24) 2024-04-27T07:00:07.643 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: basedate set to 2022-02-04 2024-04-27T07:00:07.643 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: gps base set to 2022-02-06 (week 2196) 2024-04-27T07:00:07.643 INFO:teuthology.orchestra.run.smithi053.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-27T07:00:07.643 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-27T07:00:07.643 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: restrict ::: KOD does nothing without LIMITED. 2024-04-27T07:00:07.644 INFO:teuthology.orchestra.run.smithi053.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-27T07:00:07.644 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: Listen and drop on 0 v6wildcard [::]:123 2024-04-27T07:00:07.644 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-27T07:00:07.645 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: Listen normally on 2 lo 127.0.0.1:123 2024-04-27T07:00:07.645 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: Listen normally on 3 ens1f0 172.21.15.53:123 2024-04-27T07:00:07.645 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: Listen normally on 4 lo [::1]:123 2024-04-27T07:00:07.645 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:3718%4]:123 2024-04-27T07:00:07.645 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:07 ntpd[17118]: Listening on routing socket on fd #22 for interface updates 2024-04-27T07:00:08.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:08.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:08.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:08.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:08.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:08.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:08.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:08.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:08.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:08 ntpd[17133]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:08.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:08 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:08.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:08 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:08.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:08 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:08.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:08 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:08.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:08 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:08.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:08 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:08.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:08 ntpd[17118]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:08.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:08 ntpd[17118]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:09.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:09 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:09.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:09 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:09.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:09 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:09.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:09 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:09.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:09 ntpd[17133]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:09.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:09 ntpd[17133]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:09.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:09 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:09.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:09 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:09.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:09 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:09.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:09 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:09.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:09 ntpd[17118]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:09.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:09 ntpd[17118]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:10.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:10.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:10 ntpd[17133]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:10 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:10 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:10 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:10 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:10.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:10 ntpd[17118]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:10.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:10 ntpd[17118]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:11.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:11 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:11.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:11 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:11.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:11 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:11.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:11 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:11.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:11 ntpd[17133]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:11.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:11 ntpd[17133]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:11.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:11.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:11 ntpd[17118]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:12 ntpd[17133]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:12 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:12 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:12 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:12 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:12 ntpd[17118]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:12.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:12 ntpd[17118]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:13 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:13 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:13 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:13 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:13 ntpd[17133]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:13.614 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:13 ntpd[17133]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:13.642 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:13.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:13 ntpd[17118]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:14.615 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:14 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:14.616 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:14 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:14.616 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:14 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:14.616 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:14 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:14.616 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:14 ntpd[17133]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:14.616 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:14 ntpd[17133]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:14.617 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:14 ntpd[17133]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:14.617 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:14 ntpd[17133]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:27 Apr 07:00:14 ntpd[17133]: ntpd: time slew +0.001795 s 2024-04-27T07:00:14.618 INFO:teuthology.orchestra.run.smithi012.stdout:ntpd: time slew +0.001795s 2024-04-27T07:00:14.619 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:14 ntpd[17133]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-27T07:00:14.619 INFO:teuthology.orchestra.run.smithi012.stderr:27 Apr 07:00:14 ntpd[17133]: can't open /var/log/ntpstats/loopstats.20240427: Permission denied 2024-04-27T07:00:14.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:14 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:14.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:14 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:14.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:14 ntpd[17118]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-27T07:00:14.643 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:14 ntpd[17118]: can't open /var/log/ntpstats/rawstats.20240427: Permission denied 2024-04-27T07:00:14.644 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:14 ntpd[17118]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-27T07:00:14.644 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:14 ntpd[17118]: can't open /var/log/ntpstats/peerstats.20240427: Permission denied 2024-04-27T07:00:14.644 INFO:teuthology.orchestra.run.smithi053.stdout:27 Apr 07:00:14 ntpd[17118]: ntpd: time slew -0.000309 s 2024-04-27T07:00:14.644 INFO:teuthology.orchestra.run.smithi053.stdout:ntpd: time slew -0.000309s 2024-04-27T07:00:14.645 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:14 ntpd[17118]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-27T07:00:14.645 INFO:teuthology.orchestra.run.smithi053.stderr:27 Apr 07:00:14 ntpd[17118]: can't open /var/log/ntpstats/loopstats.20240427: Permission denied 2024-04-27T07:00:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2024-04-27T07:00:14.671 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2024-04-27T07:00:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:00:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:00:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:00:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:00:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: remote refid st t when poll reach delay offset jitter 2024-04-27T07:00:14.697 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================== 2024-04-27T07:00:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:00:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:00:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:00:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:00:14.698 INFO:teuthology.run_tasks:Running task install... 2024-04-27T07:00:14.712 DEBUG:teuthology.task.install:project ceph 2024-04-27T07:00:14.712 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'}} 2024-04-27T07:00:14.712 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'} 2024-04-27T07:00:14.712 INFO:teuthology.task.install:Using flavor: default 2024-04-27T07:00:14.724 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-27T07:00:14.724 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-27T07:00:14.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-key list | grep Ceph 2024-04-27T07:00:14.726 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-key list | grep Ceph 2024-04-27T07:00:14.786 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-27T07:00:14.803 INFO:teuthology.orchestra.run.smithi053.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-27T07:00:14.814 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-27T07:00:14.814 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph.com (release key) 2024-04-27T07:00:14.815 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-27T07:00:14.815 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-27T07:00:14.816 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-27T07:00:14.831 INFO:teuthology.orchestra.run.smithi053.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-27T07:00:14.831 INFO:teuthology.orchestra.run.smithi053.stdout:uid [ unknown] Ceph.com (release key) 2024-04-27T07:00:14.831 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-27T07:00:14.832 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-27T07:00:14.832 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-27T07:00:14.972 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default/ 2024-04-27T07:00:14.972 INFO:teuthology.task.install.deb:Package version is 19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:00:14.977 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default/ 2024-04-27T07:00:14.978 INFO:teuthology.task.install.deb:Package version is 19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:00:15.290 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:00:15.290 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-27T07:00:15.293 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:00:15.293 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-27T07:00:15.304 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-04-27T07:00:15.308 DEBUG:teuthology.orchestra.run.smithi053:> sudo apt-get update 2024-04-27T07:00:15.480 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T07:00:15.495 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-27T07:00:15.516 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-27T07:00:15.597 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-27T07:00:15.704 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy InRelease 2024-04-27T07:00:15.707 INFO:teuthology.orchestra.run.smithi053.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy InRelease 2024-04-27T07:00:15.770 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-27T07:00:15.774 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-27T07:00:15.856 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-27T07:00:15.858 INFO:teuthology.orchestra.run.smithi053.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-27T07:00:15.933 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-04-27T07:00:15.935 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-04-27T07:00:15.981 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,426 B] 2024-04-27T07:00:15.988 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,426 B] 2024-04-27T07:00:16.079 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 29.3 kB in 1s (51.2 kB/s) 2024-04-27T07:00:17.077 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-27T07:00:17.084 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-27T07:00:17.084 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-27T07:00:17.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3227-gc37f63b3-1jammy cephadm=19.0.0-3227-gc37f63b3-1jammy ceph-mds=19.0.0-3227-gc37f63b3-1jammy ceph-mgr=19.0.0-3227-gc37f63b3-1jammy ceph-common=19.0.0-3227-gc37f63b3-1jammy ceph-fuse=19.0.0-3227-gc37f63b3-1jammy ceph-test=19.0.0-3227-gc37f63b3-1jammy ceph-volume=19.0.0-3227-gc37f63b3-1jammy radosgw=19.0.0-3227-gc37f63b3-1jammy python3-rados=19.0.0-3227-gc37f63b3-1jammy python3-rgw=19.0.0-3227-gc37f63b3-1jammy python3-cephfs=19.0.0-3227-gc37f63b3-1jammy python3-rbd=19.0.0-3227-gc37f63b3-1jammy libcephfs2=19.0.0-3227-gc37f63b3-1jammy libcephfs-dev=19.0.0-3227-gc37f63b3-1jammy librados2=19.0.0-3227-gc37f63b3-1jammy librbd1=19.0.0-3227-gc37f63b3-1jammy rbd-fuse=19.0.0-3227-gc37f63b3-1jammy python3-cephfs=19.0.0-3227-gc37f63b3-1jammy cephfs-shell=19.0.0-3227-gc37f63b3-1jammy cephfs-top=19.0.0-3227-gc37f63b3-1jammy cephfs-mirror=19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:00:17.094 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-27T07:00:17.113 INFO:teuthology.orchestra.run.smithi053.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-27T07:00:17.155 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-27T07:00:17.357 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-04-27T07:00:17.357 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-04-27T07:00:17.470 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 29.3 kB in 2s (14.8 kB/s) 2024-04-27T07:00:17.562 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:00:17.562 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:00:17.562 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:00:17.562 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:00:17.563 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T07:00:17.563 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:00:17.563 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:00:17.564 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-04-27T07:00:17.564 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-27T07:00:17.564 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-27T07:00:17.564 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-27T07:00:17.564 INFO:teuthology.orchestra.run.smithi012.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-27T07:00:17.565 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-27T07:00:17.565 INFO:teuthology.orchestra.run.smithi012.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-27T07:00:17.565 INFO:teuthology.orchestra.run.smithi012.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-27T07:00:17.566 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-27T07:00:17.567 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2024-04-27T07:00:17.568 INFO:teuthology.orchestra.run.smithi012.stdout: btrfs-tools 2024-04-27T07:00:17.678 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-04-27T07:00:17.678 INFO:teuthology.orchestra.run.smithi012.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-27T07:00:17.678 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-27T07:00:17.678 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-27T07:00:17.678 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-27T07:00:17.680 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-27T07:00:17.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-27T07:00:17.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-27T07:00:17.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-27T07:00:17.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-27T07:00:17.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-27T07:00:17.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-27T07:00:17.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-27T07:00:17.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-27T07:00:17.681 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-27T07:00:17.681 INFO:teuthology.orchestra.run.smithi012.stdout: socat unzip xmlstarlet zip 2024-04-27T07:00:17.931 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-27T07:00:17.931 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 182 MB of archives. 2024-04-27T07:00:17.931 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 831 MB of additional disk space will be used. 2024-04-27T07:00:17.931 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3227-gc37f63b3-1jammy [3,556 kB] 2024-04-27T07:00:18.130 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-27T07:00:18.153 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-27T07:00:18.164 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-27T07:00:18.215 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-27T07:00:18.241 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-27T07:00:18.244 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-27T07:00:18.258 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-27T07:00:18.316 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-27T07:00:18.320 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-27T07:00:18.332 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-27T07:00:18.336 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-27T07:00:18.337 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-27T07:00:18.346 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-27T07:00:18.348 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-27T07:00:18.350 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-27T07:00:18.352 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-27T07:00:18.352 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-27T07:00:18.362 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-27T07:00:18.364 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-27T07:00:18.365 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-27T07:00:18.366 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-27T07:00:18.382 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-27T07:00:18.384 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-27T07:00:18.384 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-27T07:00:18.385 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-27T07:00:18.385 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-27T07:00:18.385 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-27T07:00:18.402 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-27T07:00:18.403 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-27T07:00:18.405 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-27T07:00:18.406 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-27T07:00:18.423 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-27T07:00:18.423 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-27T07:00:18.424 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-27T07:00:18.424 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-27T07:00:18.424 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-27T07:00:18.425 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-27T07:00:18.443 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-27T07:00:18.444 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-27T07:00:18.449 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-27T07:00:18.449 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-27T07:00:18.449 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-04-27T07:00:18.463 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-27T07:00:18.464 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-27T07:00:18.466 DEBUG:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3227-gc37f63b3-1jammy cephadm=19.0.0-3227-gc37f63b3-1jammy ceph-mds=19.0.0-3227-gc37f63b3-1jammy ceph-mgr=19.0.0-3227-gc37f63b3-1jammy ceph-common=19.0.0-3227-gc37f63b3-1jammy ceph-fuse=19.0.0-3227-gc37f63b3-1jammy ceph-test=19.0.0-3227-gc37f63b3-1jammy ceph-volume=19.0.0-3227-gc37f63b3-1jammy radosgw=19.0.0-3227-gc37f63b3-1jammy python3-rados=19.0.0-3227-gc37f63b3-1jammy python3-rgw=19.0.0-3227-gc37f63b3-1jammy python3-cephfs=19.0.0-3227-gc37f63b3-1jammy python3-rbd=19.0.0-3227-gc37f63b3-1jammy libcephfs2=19.0.0-3227-gc37f63b3-1jammy libcephfs-dev=19.0.0-3227-gc37f63b3-1jammy librados2=19.0.0-3227-gc37f63b3-1jammy librbd1=19.0.0-3227-gc37f63b3-1jammy rbd-fuse=19.0.0-3227-gc37f63b3-1jammy python3-cephfs=19.0.0-3227-gc37f63b3-1jammy cephfs-shell=19.0.0-3227-gc37f63b3-1jammy cephfs-top=19.0.0-3227-gc37f63b3-1jammy cephfs-mirror=19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:00:18.466 INFO:teuthology.orchestra.run.smithi012.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-27T07:00:18.467 INFO:teuthology.orchestra.run.smithi012.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-27T07:00:18.468 INFO:teuthology.orchestra.run.smithi012.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-27T07:00:18.470 INFO:teuthology.orchestra.run.smithi012.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-27T07:00:18.484 INFO:teuthology.orchestra.run.smithi012.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-27T07:00:18.486 INFO:teuthology.orchestra.run.smithi012.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-27T07:00:18.487 INFO:teuthology.orchestra.run.smithi012.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-27T07:00:18.488 INFO:teuthology.orchestra.run.smithi012.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-27T07:00:18.505 INFO:teuthology.orchestra.run.smithi012.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-27T07:00:18.528 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-04-27T07:00:18.559 INFO:teuthology.orchestra.run.smithi012.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-27T07:00:18.560 INFO:teuthology.orchestra.run.smithi012.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-27T07:00:18.561 INFO:teuthology.orchestra.run.smithi012.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-27T07:00:18.561 INFO:teuthology.orchestra.run.smithi012.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-27T07:00:18.562 INFO:teuthology.orchestra.run.smithi012.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-27T07:00:18.585 INFO:teuthology.orchestra.run.smithi012.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-27T07:00:18.588 INFO:teuthology.orchestra.run.smithi012.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-27T07:00:18.588 INFO:teuthology.orchestra.run.smithi012.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-27T07:00:18.610 INFO:teuthology.orchestra.run.smithi012.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-27T07:00:18.610 INFO:teuthology.orchestra.run.smithi012.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-27T07:00:18.616 INFO:teuthology.orchestra.run.smithi012.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-27T07:00:18.617 INFO:teuthology.orchestra.run.smithi012.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-27T07:00:18.618 INFO:teuthology.orchestra.run.smithi012.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-27T07:00:18.618 INFO:teuthology.orchestra.run.smithi012.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-27T07:00:18.619 INFO:teuthology.orchestra.run.smithi012.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-27T07:00:18.619 INFO:teuthology.orchestra.run.smithi012.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-27T07:00:18.623 INFO:teuthology.orchestra.run.smithi012.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-27T07:00:18.637 INFO:teuthology.orchestra.run.smithi012.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-27T07:00:18.638 INFO:teuthology.orchestra.run.smithi012.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-27T07:00:18.639 INFO:teuthology.orchestra.run.smithi012.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-27T07:00:18.657 INFO:teuthology.orchestra.run.smithi012.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-27T07:00:18.660 INFO:teuthology.orchestra.run.smithi012.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-27T07:00:18.660 INFO:teuthology.orchestra.run.smithi012.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-27T07:00:18.661 INFO:teuthology.orchestra.run.smithi012.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-27T07:00:18.662 INFO:teuthology.orchestra.run.smithi012.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-27T07:00:18.666 INFO:teuthology.orchestra.run.smithi012.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-27T07:00:18.667 INFO:teuthology.orchestra.run.smithi012.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-27T07:00:18.678 INFO:teuthology.orchestra.run.smithi012.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-27T07:00:18.679 INFO:teuthology.orchestra.run.smithi012.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-27T07:00:18.679 INFO:teuthology.orchestra.run.smithi012.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-27T07:00:18.699 INFO:teuthology.orchestra.run.smithi012.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-27T07:00:18.700 INFO:teuthology.orchestra.run.smithi012.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-27T07:00:18.701 INFO:teuthology.orchestra.run.smithi012.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-27T07:00:18.708 INFO:teuthology.orchestra.run.smithi012.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-27T07:00:18.709 INFO:teuthology.orchestra.run.smithi012.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-27T07:00:18.709 INFO:teuthology.orchestra.run.smithi012.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-27T07:00:18.711 INFO:teuthology.orchestra.run.smithi012.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-27T07:00:18.719 INFO:teuthology.orchestra.run.smithi012.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-27T07:00:18.731 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2024-04-27T07:00:18.732 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2024-04-27T07:00:18.938 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:00:18.938 INFO:teuthology.orchestra.run.smithi053.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:00:18.938 INFO:teuthology.orchestra.run.smithi053.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:00:18.938 INFO:teuthology.orchestra.run.smithi053.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:00:18.938 INFO:teuthology.orchestra.run.smithi053.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T07:00:18.939 INFO:teuthology.orchestra.run.smithi053.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:00:18.939 INFO:teuthology.orchestra.run.smithi053.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:00:18.939 INFO:teuthology.orchestra.run.smithi053.stdout:The following additional packages will be installed: 2024-04-27T07:00:18.939 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-27T07:00:18.940 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-27T07:00:18.940 INFO:teuthology.orchestra.run.smithi053.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-27T07:00:18.940 INFO:teuthology.orchestra.run.smithi053.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-27T07:00:18.941 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-27T07:00:18.941 INFO:teuthology.orchestra.run.smithi053.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-27T07:00:18.941 INFO:teuthology.orchestra.run.smithi053.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-27T07:00:18.941 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-27T07:00:18.941 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-27T07:00:18.941 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-27T07:00:18.941 INFO:teuthology.orchestra.run.smithi053.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-27T07:00:18.942 INFO:teuthology.orchestra.run.smithi053.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-27T07:00:18.943 INFO:teuthology.orchestra.run.smithi053.stdout:Suggested packages: 2024-04-27T07:00:18.943 INFO:teuthology.orchestra.run.smithi053.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-27T07:00:18.943 INFO:teuthology.orchestra.run.smithi053.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-27T07:00:18.943 INFO:teuthology.orchestra.run.smithi053.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-27T07:00:18.943 INFO:teuthology.orchestra.run.smithi053.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-27T07:00:18.943 INFO:teuthology.orchestra.run.smithi053.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-27T07:00:18.943 INFO:teuthology.orchestra.run.smithi053.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-27T07:00:18.943 INFO:teuthology.orchestra.run.smithi053.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-27T07:00:18.943 INFO:teuthology.orchestra.run.smithi053.stdout:Recommended packages: 2024-04-27T07:00:18.943 INFO:teuthology.orchestra.run.smithi053.stdout: btrfs-tools 2024-04-27T07:00:19.055 INFO:teuthology.orchestra.run.smithi053.stdout:The following NEW packages will be installed: 2024-04-27T07:00:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-27T07:00:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-27T07:00:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-27T07:00:19.055 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-27T07:00:19.057 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-27T07:00:19.058 INFO:teuthology.orchestra.run.smithi053.stdout: socat unzip xmlstarlet zip 2024-04-27T07:00:19.125 INFO:teuthology.orchestra.run.smithi053.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-27T07:00:19.125 INFO:teuthology.orchestra.run.smithi053.stdout:Need to get 182 MB of archives. 2024-04-27T07:00:19.125 INFO:teuthology.orchestra.run.smithi053.stdout:After this operation, 831 MB of additional disk space will be used. 2024-04-27T07:00:19.126 INFO:teuthology.orchestra.run.smithi053.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-27T07:00:19.147 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-27T07:00:19.162 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-27T07:00:19.216 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-27T07:00:19.250 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-27T07:00:19.251 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-27T07:00:19.259 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-27T07:00:19.322 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-27T07:00:19.327 INFO:teuthology.orchestra.run.smithi053.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-27T07:00:19.339 INFO:teuthology.orchestra.run.smithi053.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-27T07:00:19.344 INFO:teuthology.orchestra.run.smithi053.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-27T07:00:19.344 INFO:teuthology.orchestra.run.smithi053.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-27T07:00:19.353 INFO:teuthology.orchestra.run.smithi053.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-27T07:00:19.355 INFO:teuthology.orchestra.run.smithi053.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-27T07:00:19.358 INFO:teuthology.orchestra.run.smithi053.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-27T07:00:19.359 INFO:teuthology.orchestra.run.smithi053.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-27T07:00:19.359 INFO:teuthology.orchestra.run.smithi053.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-27T07:00:19.376 INFO:teuthology.orchestra.run.smithi053.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-27T07:00:19.378 INFO:teuthology.orchestra.run.smithi053.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-27T07:00:19.380 INFO:teuthology.orchestra.run.smithi053.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-27T07:00:19.380 INFO:teuthology.orchestra.run.smithi053.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-27T07:00:19.397 INFO:teuthology.orchestra.run.smithi053.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-27T07:00:19.398 INFO:teuthology.orchestra.run.smithi053.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-27T07:00:19.398 INFO:teuthology.orchestra.run.smithi053.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-27T07:00:19.398 INFO:teuthology.orchestra.run.smithi053.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-27T07:00:19.399 INFO:teuthology.orchestra.run.smithi053.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-27T07:00:19.399 INFO:teuthology.orchestra.run.smithi053.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-27T07:00:19.417 INFO:teuthology.orchestra.run.smithi053.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-27T07:00:19.417 INFO:teuthology.orchestra.run.smithi053.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-27T07:00:19.420 INFO:teuthology.orchestra.run.smithi053.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-27T07:00:19.421 INFO:teuthology.orchestra.run.smithi053.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-27T07:00:19.438 INFO:teuthology.orchestra.run.smithi053.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-27T07:00:19.438 INFO:teuthology.orchestra.run.smithi053.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-27T07:00:19.439 INFO:teuthology.orchestra.run.smithi053.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-27T07:00:19.439 INFO:teuthology.orchestra.run.smithi053.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-27T07:00:19.440 INFO:teuthology.orchestra.run.smithi053.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-27T07:00:19.441 INFO:teuthology.orchestra.run.smithi053.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-27T07:00:19.458 INFO:teuthology.orchestra.run.smithi053.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-27T07:00:19.459 INFO:teuthology.orchestra.run.smithi053.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-27T07:00:19.464 INFO:teuthology.orchestra.run.smithi053.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-27T07:00:19.464 INFO:teuthology.orchestra.run.smithi053.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-27T07:00:19.479 INFO:teuthology.orchestra.run.smithi053.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-27T07:00:19.480 INFO:teuthology.orchestra.run.smithi053.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-27T07:00:19.481 INFO:teuthology.orchestra.run.smithi053.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-27T07:00:19.483 INFO:teuthology.orchestra.run.smithi053.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-27T07:00:19.484 INFO:teuthology.orchestra.run.smithi053.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-27T07:00:19.486 INFO:teuthology.orchestra.run.smithi053.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-27T07:00:19.500 INFO:teuthology.orchestra.run.smithi053.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-27T07:00:19.502 INFO:teuthology.orchestra.run.smithi053.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-27T07:00:19.503 INFO:teuthology.orchestra.run.smithi053.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-27T07:00:19.505 INFO:teuthology.orchestra.run.smithi053.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-27T07:00:19.524 INFO:teuthology.orchestra.run.smithi053.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3227-gc37f63b3-1jammy [3,556 kB] 2024-04-27T07:00:19.527 INFO:teuthology.orchestra.run.smithi053.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-27T07:00:19.585 INFO:teuthology.orchestra.run.smithi053.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-27T07:00:19.585 INFO:teuthology.orchestra.run.smithi053.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-27T07:00:19.587 INFO:teuthology.orchestra.run.smithi053.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-27T07:00:19.587 INFO:teuthology.orchestra.run.smithi053.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-27T07:00:19.589 INFO:teuthology.orchestra.run.smithi053.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-27T07:00:19.609 INFO:teuthology.orchestra.run.smithi053.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-27T07:00:19.611 INFO:teuthology.orchestra.run.smithi053.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-27T07:00:19.611 INFO:teuthology.orchestra.run.smithi053.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-27T07:00:19.619 INFO:teuthology.orchestra.run.smithi012.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3227-gc37f63b3-1jammy [3,224 kB] 2024-04-27T07:00:19.650 INFO:teuthology.orchestra.run.smithi053.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-27T07:00:19.651 INFO:teuthology.orchestra.run.smithi053.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-27T07:00:19.658 INFO:teuthology.orchestra.run.smithi053.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-27T07:00:19.659 INFO:teuthology.orchestra.run.smithi053.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-27T07:00:19.659 INFO:teuthology.orchestra.run.smithi053.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-27T07:00:19.660 INFO:teuthology.orchestra.run.smithi053.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-27T07:00:19.660 INFO:teuthology.orchestra.run.smithi053.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-27T07:00:19.661 INFO:teuthology.orchestra.run.smithi053.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-27T07:00:19.665 INFO:teuthology.orchestra.run.smithi053.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-27T07:00:19.678 INFO:teuthology.orchestra.run.smithi053.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-27T07:00:19.680 INFO:teuthology.orchestra.run.smithi053.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-27T07:00:19.681 INFO:teuthology.orchestra.run.smithi053.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-27T07:00:19.699 INFO:teuthology.orchestra.run.smithi053.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-27T07:00:19.702 INFO:teuthology.orchestra.run.smithi053.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-27T07:00:19.702 INFO:teuthology.orchestra.run.smithi053.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-27T07:00:19.704 INFO:teuthology.orchestra.run.smithi053.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-27T07:00:19.704 INFO:teuthology.orchestra.run.smithi053.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-27T07:00:19.711 INFO:teuthology.orchestra.run.smithi053.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-27T07:00:19.711 INFO:teuthology.orchestra.run.smithi053.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-27T07:00:19.732 INFO:teuthology.orchestra.run.smithi053.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-27T07:00:19.733 INFO:teuthology.orchestra.run.smithi053.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-27T07:00:19.734 INFO:teuthology.orchestra.run.smithi053.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-27T07:00:19.753 INFO:teuthology.orchestra.run.smithi053.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-27T07:00:19.755 INFO:teuthology.orchestra.run.smithi053.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-27T07:00:19.756 INFO:teuthology.orchestra.run.smithi053.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-27T07:00:19.765 INFO:teuthology.orchestra.run.smithi053.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-27T07:00:19.765 INFO:teuthology.orchestra.run.smithi053.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-27T07:00:19.766 INFO:teuthology.orchestra.run.smithi053.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-27T07:00:19.769 INFO:teuthology.orchestra.run.smithi053.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-27T07:00:19.773 INFO:teuthology.orchestra.run.smithi053.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-27T07:00:20.309 INFO:teuthology.orchestra.run.smithi012.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3227-gc37f63b3-1jammy [712 kB] 2024-04-27T07:00:20.437 INFO:teuthology.orchestra.run.smithi012.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3227-gc37f63b3-1jammy [356 kB] 2024-04-27T07:00:20.492 INFO:teuthology.orchestra.run.smithi012.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3227-gc37f63b3-1jammy [32.8 kB] 2024-04-27T07:00:20.498 INFO:teuthology.orchestra.run.smithi012.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3227-gc37f63b3-1jammy [180 kB] 2024-04-27T07:00:20.529 INFO:teuthology.orchestra.run.smithi012.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3227-gc37f63b3-1jammy [67.1 kB] 2024-04-27T07:00:20.542 INFO:teuthology.orchestra.run.smithi012.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3227-gc37f63b3-1jammy [330 kB] 2024-04-27T07:00:20.590 INFO:teuthology.orchestra.run.smithi012.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3227-gc37f63b3-1jammy [6,893 kB] 2024-04-27T07:00:21.469 INFO:teuthology.orchestra.run.smithi053.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3227-gc37f63b3-1jammy [3,224 kB] 2024-04-27T07:00:21.536 INFO:teuthology.orchestra.run.smithi012.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3227-gc37f63b3-1jammy [112 kB] 2024-04-27T07:00:21.548 INFO:teuthology.orchestra.run.smithi012.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3227-gc37f63b3-1jammy [466 kB] 2024-04-27T07:00:21.625 INFO:teuthology.orchestra.run.smithi012.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3227-gc37f63b3-1jammy [25.8 MB] 2024-04-27T07:00:23.032 INFO:teuthology.orchestra.run.smithi053.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3227-gc37f63b3-1jammy [712 kB] 2024-04-27T07:00:23.404 INFO:teuthology.orchestra.run.smithi053.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3227-gc37f63b3-1jammy [356 kB] 2024-04-27T07:00:23.579 INFO:teuthology.orchestra.run.smithi053.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3227-gc37f63b3-1jammy [32.8 kB] 2024-04-27T07:00:23.581 INFO:teuthology.orchestra.run.smithi053.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3227-gc37f63b3-1jammy [180 kB] 2024-04-27T07:00:23.664 INFO:teuthology.orchestra.run.smithi053.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3227-gc37f63b3-1jammy [67.1 kB] 2024-04-27T07:00:23.668 INFO:teuthology.orchestra.run.smithi053.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3227-gc37f63b3-1jammy [330 kB] 2024-04-27T07:00:23.825 INFO:teuthology.orchestra.run.smithi053.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3227-gc37f63b3-1jammy [6,893 kB] 2024-04-27T07:00:26.491 INFO:teuthology.orchestra.run.smithi012.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3227-gc37f63b3-1jammy [5,092 kB] 2024-04-27T07:00:26.941 INFO:teuthology.orchestra.run.smithi053.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3227-gc37f63b3-1jammy [112 kB] 2024-04-27T07:00:27.007 INFO:teuthology.orchestra.run.smithi053.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3227-gc37f63b3-1jammy [466 kB] 2024-04-27T07:00:27.174 INFO:teuthology.orchestra.run.smithi053.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3227-gc37f63b3-1jammy [25.8 MB] 2024-04-27T07:00:27.358 INFO:teuthology.orchestra.run.smithi012.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3227-gc37f63b3-1jammy [244 kB] 2024-04-27T07:00:27.412 INFO:teuthology.orchestra.run.smithi012.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3227-gc37f63b3-1jammy [124 kB] 2024-04-27T07:00:27.422 INFO:teuthology.orchestra.run.smithi012.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3227-gc37f63b3-1jammy [1,526 kB] 2024-04-27T07:00:27.680 INFO:teuthology.orchestra.run.smithi012.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3227-gc37f63b3-1jammy [6,257 kB] 2024-04-27T07:00:28.740 INFO:teuthology.orchestra.run.smithi012.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3227-gc37f63b3-1jammy [22.8 MB] 2024-04-27T07:00:32.860 INFO:teuthology.orchestra.run.smithi012.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3227-gc37f63b3-1jammy [14.1 kB] 2024-04-27T07:00:32.919 INFO:teuthology.orchestra.run.smithi012.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3227-gc37f63b3-1jammy [870 kB] 2024-04-27T07:00:33.095 INFO:teuthology.orchestra.run.smithi012.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3227-gc37f63b3-1jammy [2,437 kB] 2024-04-27T07:00:33.602 INFO:teuthology.orchestra.run.smithi012.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3227-gc37f63b3-1jammy [791 kB] 2024-04-27T07:00:33.793 INFO:teuthology.orchestra.run.smithi012.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3227-gc37f63b3-1jammy [148 kB] 2024-04-27T07:00:33.820 INFO:teuthology.orchestra.run.smithi012.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3227-gc37f63b3-1jammy [2,297 kB] 2024-04-27T07:00:34.471 INFO:teuthology.orchestra.run.smithi012.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3227-gc37f63b3-1jammy [8,625 kB] 2024-04-27T07:00:34.550 INFO:teuthology.orchestra.run.smithi053.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3227-gc37f63b3-1jammy [5,092 kB] 2024-04-27T07:00:35.649 INFO:teuthology.orchestra.run.smithi053.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3227-gc37f63b3-1jammy [244 kB] 2024-04-27T07:00:35.715 INFO:teuthology.orchestra.run.smithi053.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3227-gc37f63b3-1jammy [124 kB] 2024-04-27T07:00:35.727 INFO:teuthology.orchestra.run.smithi053.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3227-gc37f63b3-1jammy [1,526 kB] 2024-04-27T07:00:36.130 INFO:teuthology.orchestra.run.smithi053.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3227-gc37f63b3-1jammy [6,257 kB] 2024-04-27T07:00:37.455 INFO:teuthology.orchestra.run.smithi012.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3227-gc37f63b3-1jammy [14.2 kB] 2024-04-27T07:00:37.456 INFO:teuthology.orchestra.run.smithi012.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3227-gc37f63b3-1jammy [51.6 MB] 2024-04-27T07:00:37.755 INFO:teuthology.orchestra.run.smithi053.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3227-gc37f63b3-1jammy [22.8 MB] 2024-04-27T07:00:42.305 INFO:teuthology.orchestra.run.smithi053.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3227-gc37f63b3-1jammy [14.1 kB] 2024-04-27T07:00:42.306 INFO:teuthology.orchestra.run.smithi053.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3227-gc37f63b3-1jammy [870 kB] 2024-04-27T07:00:42.444 INFO:teuthology.orchestra.run.smithi053.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3227-gc37f63b3-1jammy [2,437 kB] 2024-04-27T07:00:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3227-gc37f63b3-1jammy [791 kB] 2024-04-27T07:00:43.078 INFO:teuthology.orchestra.run.smithi053.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3227-gc37f63b3-1jammy [148 kB] 2024-04-27T07:00:43.085 INFO:teuthology.orchestra.run.smithi053.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3227-gc37f63b3-1jammy [2,297 kB] 2024-04-27T07:00:43.509 INFO:teuthology.orchestra.run.smithi053.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3227-gc37f63b3-1jammy [8,625 kB] 2024-04-27T07:00:45.036 INFO:teuthology.orchestra.run.smithi053.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3227-gc37f63b3-1jammy [14.2 kB] 2024-04-27T07:00:45.036 INFO:teuthology.orchestra.run.smithi053.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3227-gc37f63b3-1jammy [51.6 MB] 2024-04-27T07:00:48.419 INFO:teuthology.orchestra.run.smithi012.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3227-gc37f63b3-1jammy [124 kB] 2024-04-27T07:00:48.422 INFO:teuthology.orchestra.run.smithi012.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3227-gc37f63b3-1jammy [232 kB] 2024-04-27T07:00:48.427 INFO:teuthology.orchestra.run.smithi012.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3227-gc37f63b3-1jammy [33.9 kB] 2024-04-27T07:00:48.428 INFO:teuthology.orchestra.run.smithi012.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3227-gc37f63b3-1jammy [23.4 kB] 2024-04-27T07:00:48.428 INFO:teuthology.orchestra.run.smithi012.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3227-gc37f63b3-1jammy [39.0 kB] 2024-04-27T07:00:48.430 INFO:teuthology.orchestra.run.smithi012.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3227-gc37f63b3-1jammy [13.6 MB] 2024-04-27T07:00:51.890 INFO:teuthology.orchestra.run.smithi012.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3227-gc37f63b3-1jammy [91.4 kB] 2024-04-27T07:00:52.530 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 182 MB in 34s (5,308 kB/s) 2024-04-27T07:00:52.649 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-27T07:00:52.705 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-27T07:00:52.710 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-27T07:00:52.747 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-27T07:00:52.983 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-27T07:00:53.010 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-27T07:00:53.050 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-27T07:00:53.378 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cmd2. 2024-04-27T07:00:53.405 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-27T07:00:53.444 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-27T07:00:53.823 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-27T07:00:53.850 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-27T07:00:53.931 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-27T07:00:54.226 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-27T07:00:54.242 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-27T07:00:54.276 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-27T07:00:54.621 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-27T07:00:54.648 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-27T07:00:54.688 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-27T07:00:55.032 INFO:teuthology.orchestra.run.smithi053.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3227-gc37f63b3-1jammy [124 kB] 2024-04-27T07:00:55.034 INFO:teuthology.orchestra.run.smithi053.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3227-gc37f63b3-1jammy [232 kB] 2024-04-27T07:00:55.039 INFO:teuthology.orchestra.run.smithi053.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3227-gc37f63b3-1jammy [33.9 kB] 2024-04-27T07:00:55.040 INFO:teuthology.orchestra.run.smithi053.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3227-gc37f63b3-1jammy [23.4 kB] 2024-04-27T07:00:55.041 INFO:teuthology.orchestra.run.smithi053.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3227-gc37f63b3-1jammy [39.0 kB] 2024-04-27T07:00:55.042 INFO:teuthology.orchestra.run.smithi053.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3227-gc37f63b3-1jammy [13.6 MB] 2024-04-27T07:00:55.274 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-27T07:00:55.302 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:00:55.349 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:00:55.686 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-27T07:00:55.713 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:00:55.744 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:00:56.131 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-27T07:00:56.158 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:00:56.197 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:00:56.559 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-27T07:00:56.586 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-27T07:00:56.625 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-27T07:00:56.920 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librados2. 2024-04-27T07:00:56.947 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../010-librados2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:00:56.978 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librados2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:00:57.440 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librbd1. 2024-04-27T07:00:57.466 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../011-librbd1_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:00:57.506 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librbd1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:00:57.951 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs2. 2024-04-27T07:00:57.977 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:00:58.017 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:00:58.289 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rados. 2024-04-27T07:00:58.309 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:00:58.347 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rados (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:00:58.361 INFO:teuthology.orchestra.run.smithi053.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/main/c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3227-gc37f63b3-1jammy [91.4 kB] 2024-04-27T07:00:58.681 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-27T07:00:58.702 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:00:58.741 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-argparse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:00:58.994 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cephfs. 2024-04-27T07:00:59.003 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 182 MB in 39s (4,620 kB/s) 2024-04-27T07:00:59.010 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:00:59.052 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cephfs (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:00:59.123 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-27T07:00:59.220 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-27T07:00:59.226 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-27T07:00:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-27T07:00:59.329 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-27T07:00:59.345 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:00:59.379 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:00:59.507 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-27T07:00:59.534 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-27T07:00:59.574 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-27T07:00:59.667 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-prettytable. 2024-04-27T07:00:59.682 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-27T07:00:59.716 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-27T07:00:59.902 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cmd2. 2024-04-27T07:00:59.928 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-27T07:00:59.968 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-27T07:01:00.010 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rbd. 2024-04-27T07:01:00.027 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:00.068 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rbd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:00.321 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-27T07:01:00.348 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-27T07:01:00.422 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-27T07:01:00.448 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-27T07:01:00.463 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-27T07:01:00.488 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-27T07:01:00.774 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-27T07:01:00.790 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-27T07:01:00.824 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-27T07:01:00.825 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-27T07:01:00.852 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-27T07:01:00.892 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-27T07:01:01.086 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-27T07:01:01.101 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-27T07:01:01.135 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-27T07:01:01.179 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-27T07:01:01.194 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-27T07:01:01.228 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-27T07:01:01.480 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua5.1. 2024-04-27T07:01:01.507 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-27T07:01:01.546 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-27T07:01:01.674 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-27T07:01:01.701 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:01:01.749 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:01:01.866 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-any. 2024-04-27T07:01:01.893 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-27T07:01:01.933 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-27T07:01:02.144 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-27T07:01:02.151 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package zip. 2024-04-27T07:01:02.168 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-27T07:01:02.170 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:01:02.202 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking zip (3.0-12build2) ... 2024-04-27T07:01:02.210 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:01:02.501 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package unzip. 2024-04-27T07:01:02.524 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-27T07:01:02.556 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-27T07:01:02.573 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-27T07:01:02.582 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-27T07:01:02.635 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:01:02.942 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-27T07:01:02.951 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package luarocks. 2024-04-27T07:01:02.969 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-27T07:01:02.979 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-27T07:01:03.002 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-27T07:01:03.019 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-27T07:01:03.313 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librados2. 2024-04-27T07:01:03.339 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../010-librados2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:03.378 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librados2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:03.400 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librgw2. 2024-04-27T07:01:03.428 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../027-librgw2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:03.467 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librgw2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:03.973 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librbd1. 2024-04-27T07:01:04.001 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../011-librbd1_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:04.018 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rgw. 2024-04-27T07:01:04.037 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:04.040 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librbd1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:04.077 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:04.438 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-27T07:01:04.465 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-27T07:01:04.493 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libcephfs2. 2024-04-27T07:01:04.505 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-27T07:01:04.519 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:04.576 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libcephfs2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:04.824 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libradosstriper1. 2024-04-27T07:01:04.829 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rados. 2024-04-27T07:01:04.844 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:04.851 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:04.878 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rados (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:04.891 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libradosstriper1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:05.181 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-27T07:01:05.208 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:05.248 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-ceph-argparse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:05.252 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-common. 2024-04-27T07:01:05.279 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:05.310 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:05.500 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cephfs. 2024-04-27T07:01:05.527 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:05.567 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cephfs (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:05.887 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-27T07:01:05.913 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:05.953 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:06.240 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-prettytable. 2024-04-27T07:01:06.266 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-27T07:01:06.298 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-27T07:01:06.534 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rbd. 2024-04-27T07:01:06.561 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:06.601 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rbd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:06.791 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-base. 2024-04-27T07:01:06.818 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:06.867 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-base (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:06.954 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-27T07:01:06.980 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-27T07:01:07.020 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-27T07:01:07.298 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-27T07:01:07.325 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-27T07:01:07.365 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-27T07:01:07.445 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-27T07:01:07.472 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-27T07:01:07.511 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-27T07:01:07.626 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-27T07:01:07.653 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-27T07:01:07.684 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-27T07:01:07.797 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cheroot. 2024-04-27T07:01:07.825 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-27T07:01:07.856 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-27T07:01:08.005 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package lua5.1. 2024-04-27T07:01:08.031 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-27T07:01:08.071 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-27T07:01:08.134 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-27T07:01:08.161 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-27T07:01:08.200 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-27T07:01:08.391 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package lua-any. 2024-04-27T07:01:08.418 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-27T07:01:08.458 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-27T07:01:08.503 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-27T07:01:08.530 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-27T07:01:08.570 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-27T07:01:08.668 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package zip. 2024-04-27T07:01:08.684 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-27T07:01:08.718 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking zip (3.0-12build2) ... 2024-04-27T07:01:08.873 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-27T07:01:08.900 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-27T07:01:08.939 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-27T07:01:09.064 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package unzip. 2024-04-27T07:01:09.090 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-27T07:01:09.133 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-27T07:01:09.209 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempora. 2024-04-27T07:01:09.236 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-27T07:01:09.267 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-27T07:01:09.553 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-portend. 2024-04-27T07:01:09.580 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-27T07:01:09.620 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-27T07:01:09.659 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package luarocks. 2024-04-27T07:01:09.686 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-27T07:01:09.726 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-27T07:01:09.906 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-27T07:01:09.933 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-27T07:01:09.972 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-27T07:01:10.109 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package librgw2. 2024-04-27T07:01:10.136 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../027-librgw2_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:10.175 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking librgw2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:10.275 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-27T07:01:10.303 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-27T07:01:10.342 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-27T07:01:10.728 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rgw. 2024-04-27T07:01:10.745 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-natsort. 2024-04-27T07:01:10.755 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:10.773 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-27T07:01:10.786 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:10.820 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-27T07:01:11.098 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-27T07:01:11.124 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-27T07:01:11.149 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-logutils. 2024-04-27T07:01:11.165 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-27T07:01:11.176 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-27T07:01:11.215 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-27T07:01:11.476 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libradosstriper1. 2024-04-27T07:01:11.503 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:11.518 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-27T07:01:11.542 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libradosstriper1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:11.545 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-27T07:01:11.584 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-27T07:01:11.913 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-common. 2024-04-27T07:01:11.939 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:11.996 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-mako. 2024-04-27T07:01:12.022 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-27T07:01:12.063 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:12.070 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-27T07:01:12.399 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-27T07:01:12.426 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-27T07:01:12.465 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-27T07:01:12.735 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-27T07:01:12.762 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-27T07:01:12.801 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-27T07:01:13.096 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webob. 2024-04-27T07:01:13.123 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-27T07:01:13.162 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-27T07:01:13.428 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-base. 2024-04-27T07:01:13.459 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:13.482 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-waitress. 2024-04-27T07:01:13.504 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-base (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:13.509 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-27T07:01:13.552 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-27T07:01:13.843 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempita. 2024-04-27T07:01:13.870 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-27T07:01:13.910 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-27T07:01:14.057 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-27T07:01:14.084 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-27T07:01:14.124 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-27T07:01:14.188 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-paste. 2024-04-27T07:01:14.215 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-27T07:01:14.254 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-27T07:01:14.411 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cheroot. 2024-04-27T07:01:14.437 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-27T07:01:14.480 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-27T07:01:14.576 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-27T07:01:14.595 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-27T07:01:14.633 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-27T07:01:14.771 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-27T07:01:14.787 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-27T07:01:14.821 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-27T07:01:14.935 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-27T07:01:14.962 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-27T07:01:15.002 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-27T07:01:15.124 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-27T07:01:15.151 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-27T07:01:15.191 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-27T07:01:15.321 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webtest. 2024-04-27T07:01:15.348 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-27T07:01:15.388 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-27T07:01:15.510 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-27T07:01:15.537 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-27T07:01:15.577 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-27T07:01:15.649 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pecan. 2024-04-27T07:01:15.665 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-27T07:01:15.711 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-27T07:01:15.855 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-tempora. 2024-04-27T07:01:15.881 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-27T07:01:15.921 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-27T07:01:16.087 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-certifi. 2024-04-27T07:01:16.114 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-27T07:01:16.154 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-27T07:01:16.174 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-portend. 2024-04-27T07:01:16.201 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-27T07:01:16.240 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-27T07:01:16.527 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-27T07:01:16.554 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-27T07:01:16.594 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-27T07:01:16.682 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-idna. 2024-04-27T07:01:16.709 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-27T07:01:16.740 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-27T07:01:16.913 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-27T07:01:16.943 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-27T07:01:16.979 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-27T07:01:17.034 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-urllib3. 2024-04-27T07:01:17.062 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-27T07:01:17.101 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-27T07:01:17.324 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-natsort. 2024-04-27T07:01:17.340 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-27T07:01:17.382 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-27T07:01:17.421 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests. 2024-04-27T07:01:17.448 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-27T07:01:17.487 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-27T07:01:17.686 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-logutils. 2024-04-27T07:01:17.713 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-27T07:01:17.752 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-27T07:01:17.807 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-27T07:01:17.834 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-27T07:01:17.873 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-27T07:01:18.005 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-27T07:01:18.021 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-27T07:01:18.055 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-27T07:01:18.143 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-27T07:01:18.159 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:18.201 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:18.358 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-mako. 2024-04-27T07:01:18.386 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-27T07:01:18.424 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-27T07:01:18.590 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-27T07:01:18.617 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:18.656 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:18.770 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-27T07:01:18.797 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-27T07:01:18.837 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-27T07:01:18.919 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-27T07:01:18.945 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-27T07:01:18.997 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-27T07:01:19.156 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-27T07:01:19.200 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-27T07:01:19.222 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-27T07:01:19.304 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-openssl. 2024-04-27T07:01:19.330 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-27T07:01:19.371 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-27T07:01:19.667 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-webob. 2024-04-27T07:01:19.693 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-27T07:01:19.715 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr. 2024-04-27T07:01:19.725 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-27T07:01:19.743 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:19.781 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:20.061 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-waitress. 2024-04-27T07:01:20.088 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-27T07:01:20.132 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-27T07:01:20.152 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mon. 2024-04-27T07:01:20.178 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:20.209 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mon (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:20.439 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-tempita. 2024-04-27T07:01:20.466 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-27T07:01:20.505 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-27T07:01:20.788 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-osd. 2024-04-27T07:01:20.809 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-paste. 2024-04-27T07:01:20.815 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:20.836 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-27T07:01:20.854 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-osd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:20.875 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-27T07:01:21.187 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-27T07:01:21.203 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-27T07:01:21.245 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-27T07:01:21.531 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-27T07:01:21.557 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-27T07:01:21.597 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-27T07:01:21.900 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-webtest. 2024-04-27T07:01:21.927 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-27T07:01:21.949 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph. 2024-04-27T07:01:21.965 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../068-ceph_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:21.966 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-27T07:01:22.007 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:22.286 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pecan. 2024-04-27T07:01:22.313 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-27T07:01:22.326 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-fuse. 2024-04-27T07:01:22.353 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-27T07:01:22.353 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:22.393 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:22.716 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-certifi. 2024-04-27T07:01:22.721 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mds. 2024-04-27T07:01:22.743 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-27T07:01:22.748 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:22.782 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-27T07:01:22.787 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mds (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:23.077 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-idna. 2024-04-27T07:01:23.104 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-27T07:01:23.143 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-27T07:01:23.199 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephadm. 2024-04-27T07:01:23.226 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../071-cephadm_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:23.265 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:23.455 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-urllib3. 2024-04-27T07:01:23.482 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-27T07:01:23.521 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-27T07:01:23.585 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-27T07:01:23.612 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-27T07:01:23.651 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-27T07:01:23.841 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-requests. 2024-04-27T07:01:23.868 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-27T07:01:23.908 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-27T07:01:23.970 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-27T07:01:23.998 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-27T07:01:24.037 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-27T07:01:24.219 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-27T07:01:24.246 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-27T07:01:24.285 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-27T07:01:24.572 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-27T07:01:24.588 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:24.630 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:25.068 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-27T07:01:25.096 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:25.127 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:25.142 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-babel. 2024-04-27T07:01:25.170 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-27T07:01:25.208 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-27T07:01:25.421 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-27T07:01:25.448 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-27T07:01:25.487 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-27T07:01:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jinja2. 2024-04-27T07:01:25.580 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-27T07:01:25.619 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-27T07:01:25.798 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-openssl. 2024-04-27T07:01:25.826 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-27T07:01:25.865 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-27T07:01:25.956 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-27T07:01:25.972 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:26.006 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:26.210 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr. 2024-04-27T07:01:26.237 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:26.276 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:26.317 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-27T07:01:26.345 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-27T07:01:26.384 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-27T07:01:26.645 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-routes. 2024-04-27T07:01:26.646 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mon. 2024-04-27T07:01:26.662 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:26.672 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-27T07:01:26.696 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mon (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:26.703 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-27T07:01:27.074 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-27T07:01:27.101 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:27.131 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:27.325 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-osd. 2024-04-27T07:01:27.352 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:27.391 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-osd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:28.370 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-27T07:01:28.398 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-27T07:01:28.436 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-27T07:01:28.486 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph. 2024-04-27T07:01:28.513 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../068-ceph_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:28.553 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:28.839 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-fuse. 2024-04-27T07:01:28.865 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:28.897 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:28.957 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-joblib. 2024-04-27T07:01:28.984 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-27T07:01:29.023 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-27T07:01:29.283 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mds. 2024-04-27T07:01:29.310 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:29.343 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-27T07:01:29.349 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mds (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:29.371 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-27T07:01:29.503 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-27T07:01:29.761 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package cephadm. 2024-04-27T07:01:29.788 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../071-cephadm_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:29.790 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn. 2024-04-27T07:01:29.818 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-27T07:01:29.828 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:29.856 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-27T07:01:30.281 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-27T07:01:30.297 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-27T07:01:30.338 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-27T07:01:30.540 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-27T07:01:30.566 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:30.640 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:30.649 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-27T07:01:30.674 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-27T07:01:30.707 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-27T07:01:31.593 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cachetools. 2024-04-27T07:01:31.621 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-27T07:01:31.660 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-27T07:01:31.729 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-babel. 2024-04-27T07:01:31.757 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-27T07:01:31.796 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-27T07:01:32.132 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-jinja2. 2024-04-27T07:01:32.154 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-27T07:01:32.159 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-27T07:01:32.182 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-27T07:01:32.198 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-27T07:01:32.220 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-27T07:01:32.533 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-27T07:01:32.552 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-27T07:01:32.560 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-27T07:01:32.579 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:32.599 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-27T07:01:32.627 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:32.911 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rsa. 2024-04-27T07:01:32.935 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-27T07:01:32.972 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-27T07:01:32.978 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-27T07:01:32.999 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-27T07:01:33.038 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-27T07:01:33.264 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-google-auth. 2024-04-27T07:01:33.281 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-27T07:01:33.314 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-27T07:01:33.366 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-routes. 2024-04-27T07:01:33.393 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-27T07:01:33.432 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-27T07:01:33.592 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-27T07:01:33.619 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-27T07:01:33.658 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-27T07:01:33.744 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-27T07:01:33.771 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:33.803 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:33.970 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-websocket. 2024-04-27T07:01:33.998 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-27T07:01:34.036 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-27T07:01:34.365 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-27T07:01:34.392 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-27T07:01:34.431 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-27T07:01:35.025 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-27T07:01:35.041 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-27T07:01:35.087 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-27T07:01:35.095 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-27T07:01:35.112 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:35.153 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:35.473 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-27T07:01:35.501 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-27T07:01:35.540 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-27T07:01:35.803 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-joblib. 2024-04-27T07:01:35.820 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-27T07:01:35.861 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-27T07:01:35.884 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-27T07:01:35.912 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-27T07:01:35.951 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-27T07:01:36.171 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package jq. 2024-04-27T07:01:36.186 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-27T07:01:36.199 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-27T07:01:36.220 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-27T07:01:36.225 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-27T07:01:36.350 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-27T07:01:36.458 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package socat. 2024-04-27T07:01:36.486 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-27T07:01:36.524 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-27T07:01:36.652 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-sklearn. 2024-04-27T07:01:36.680 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-27T07:01:36.727 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-27T07:01:36.844 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package xmlstarlet. 2024-04-27T07:01:36.872 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-27T07:01:36.910 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-27T07:01:37.298 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-test. 2024-04-27T07:01:37.326 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:37.364 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-test (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:37.428 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-27T07:01:37.456 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:37.495 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:38.447 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-cachetools. 2024-04-27T07:01:38.475 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-27T07:01:38.514 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-27T07:01:38.816 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-27T07:01:38.844 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-27T07:01:38.883 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-27T07:01:39.220 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-27T07:01:39.248 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-27T07:01:39.286 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-27T07:01:39.597 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-rsa. 2024-04-27T07:01:39.614 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-27T07:01:39.656 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-27T07:01:39.685 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-volume. 2024-04-27T07:01:39.713 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:39.752 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-volume (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:39.993 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-google-auth. 2024-04-27T07:01:40.021 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-27T07:01:40.059 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-27T07:01:40.114 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-27T07:01:40.142 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:40.180 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephfs-mirror (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:40.354 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-27T07:01:40.382 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-27T07:01:40.412 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-27T07:01:40.500 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-colorama. 2024-04-27T07:01:40.528 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-27T07:01:40.567 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-27T07:01:40.718 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-websocket. 2024-04-27T07:01:40.743 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-27T07:01:40.782 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-27T07:01:40.828 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephfs-shell. 2024-04-27T07:01:40.845 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:40.886 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephfs-shell (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:41.102 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-27T07:01:41.130 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-27T07:01:41.138 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephfs-top. 2024-04-27T07:01:41.155 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:41.168 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-27T07:01:41.197 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephfs-top (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:41.424 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-27T07:01:41.452 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:41.491 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs-dev (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:41.811 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-27T07:01:41.838 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-27T07:01:41.877 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-27T07:01:41.899 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-27T07:01:41.927 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:41.965 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:42.206 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-27T07:01:42.234 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-27T07:01:42.272 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-27T07:01:42.301 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-27T07:01:42.329 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-27T07:01:42.368 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-27T07:01:42.652 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package nvme-cli. 2024-04-27T07:01:42.680 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-27T07:01:42.713 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-27T07:01:42.718 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-27T07:01:42.741 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-27T07:01:42.779 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-27T07:01:43.007 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package jq. 2024-04-27T07:01:43.023 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-27T07:01:43.057 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-27T07:01:43.115 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package pkg-config. 2024-04-27T07:01:43.143 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-27T07:01:43.181 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-27T07:01:43.301 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package socat. 2024-04-27T07:01:43.325 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-27T07:01:43.367 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-27T07:01:43.401 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-27T07:01:43.417 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-27T07:01:43.451 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-27T07:01:43.687 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package xmlstarlet. 2024-04-27T07:01:43.704 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-27T07:01:43.746 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-27T07:01:43.845 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-27T07:01:43.862 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-27T07:01:43.895 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-27T07:01:44.091 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-test. 2024-04-27T07:01:44.109 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:44.149 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-test (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:44.164 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastescript. 2024-04-27T07:01:44.193 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-27T07:01:44.231 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-27T07:01:44.576 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pluggy. 2024-04-27T07:01:44.604 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-27T07:01:44.643 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-27T07:01:44.954 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-psutil. 2024-04-27T07:01:44.982 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-27T07:01:45.022 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-27T07:01:45.365 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-py. 2024-04-27T07:01:45.393 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-27T07:01:45.440 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-27T07:01:45.760 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pygments. 2024-04-27T07:01:45.788 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-27T07:01:45.826 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-27T07:01:46.339 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-27T07:01:46.367 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-27T07:01:46.405 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-27T07:01:46.537 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ceph-volume. 2024-04-27T07:01:46.563 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:46.595 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ceph-volume (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:46.675 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-toml. 2024-04-27T07:01:46.703 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-27T07:01:46.742 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-27T07:01:46.941 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-27T07:01:46.969 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:46.999 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking cephfs-mirror (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:47.044 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pytest. 2024-04-27T07:01:47.072 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-27T07:01:47.110 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-27T07:01:47.302 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-colorama. 2024-04-27T07:01:47.330 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-27T07:01:47.368 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-27T07:01:47.406 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplejson. 2024-04-27T07:01:47.434 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-27T07:01:47.464 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-27T07:01:47.630 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package cephfs-shell. 2024-04-27T07:01:47.658 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:47.696 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking cephfs-shell (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:47.940 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package cephfs-top. 2024-04-27T07:01:47.957 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3227-gc37f63b3-1jammy_all.deb ... 2024-04-27T07:01:47.998 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking cephfs-top (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:48.008 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-27T07:01:48.036 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-27T07:01:48.074 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-27T07:01:48.243 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-27T07:01:48.260 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:48.301 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libcephfs-dev (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:48.579 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-27T07:01:48.607 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-27T07:01:48.645 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-27T07:01:48.787 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package radosgw. 2024-04-27T07:01:48.815 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../122-radosgw_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:48.853 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking radosgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:48.983 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-27T07:01:49.008 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-27T07:01:49.049 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-27T07:01:49.437 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package nvme-cli. 2024-04-27T07:01:49.465 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-27T07:01:49.503 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-27T07:01:49.690 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package rbd-fuse. 2024-04-27T07:01:49.718 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:49.756 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking rbd-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:49.950 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package pkg-config. 2024-04-27T07:01:49.978 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-27T07:01:50.016 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-27T07:01:50.096 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-27T07:01:50.252 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-27T07:01:50.277 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-27T07:01:50.310 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-27T07:01:50.386 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-27T07:01:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-27T07:01:50.742 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-27T07:01:50.769 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-27T07:01:50.784 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-27T07:01:50.808 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-27T07:01:51.094 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pastescript. 2024-04-27T07:01:51.122 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-27T07:01:51.157 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-27T07:01:51.163 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-27T07:01:51.413 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pluggy. 2024-04-27T07:01:51.442 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-27T07:01:51.480 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-27T07:01:51.627 INFO:teuthology.orchestra.run.smithi012.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-27T07:01:51.680 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:51.794 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-psutil. 2024-04-27T07:01:51.820 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-27T07:01:51.858 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-27T07:01:52.186 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-py. 2024-04-27T07:01:52.189 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user cephadm....done 2024-04-27T07:01:52.213 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-27T07:01:52.252 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-27T07:01:52.336 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-27T07:01:52.564 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pygments. 2024-04-27T07:01:52.592 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-27T07:01:52.614 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-27T07:01:52.630 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-27T07:01:52.881 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-27T07:01:53.007 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-27T07:01:53.109 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-27T07:01:53.137 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-27T07:01:53.176 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-27T07:01:53.283 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-27T07:01:53.437 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-toml. 2024-04-27T07:01:53.453 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-27T07:01:53.486 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-27T07:01:53.526 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-27T07:01:53.627 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-27T07:01:53.789 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-pytest. 2024-04-27T07:01:53.818 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-27T07:01:53.856 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-27T07:01:53.945 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-27T07:01:54.192 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package python3-simplejson. 2024-04-27T07:01:54.221 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-27T07:01:54.259 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-27T07:01:54.305 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-27T07:01:54.536 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-27T07:01:54.553 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-27T07:01:54.573 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-27T07:01:54.594 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-27T07:01:54.833 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-27T07:01:55.101 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-27T07:01:55.319 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-27T07:01:55.357 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package radosgw. 2024-04-27T07:01:55.386 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../122-radosgw_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:55.423 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking radosgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:55.578 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-27T07:01:55.846 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-27T07:01:56.105 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-argparse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:56.373 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-27T07:01:56.468 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package rbd-fuse. 2024-04-27T07:01:56.485 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3227-gc37f63b3-1jammy_amd64.deb ... 2024-04-27T07:01:56.499 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-27T07:01:56.526 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking rbd-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:56.617 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-27T07:01:56.823 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-27T07:01:56.876 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-27T07:01:57.002 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-27T07:01:57.081 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-27T07:01:57.207 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-27T07:01:57.261 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-27T07:01:57.475 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-27T07:01:57.632 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-27T07:01:57.701 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-27T07:01:57.757 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-27T07:01:57.815 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-27T07:01:57.874 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-27T07:01:58.225 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-27T07:01:58.269 INFO:teuthology.orchestra.run.smithi053.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-27T07:01:58.326 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-27T07:01:58.342 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:01:58.602 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-27T07:01:58.868 INFO:teuthology.orchestra.run.smithi053.stdout:Adding system user cephadm....done 2024-04-27T07:01:58.879 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-27T07:01:59.024 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-27T07:01:59.180 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up zip (3.0-12build2) ... 2024-04-27T07:01:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-27T07:01:59.306 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-27T07:01:59.576 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-27T07:01:59.702 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-27T07:01:59.953 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-27T07:01:59.982 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-27T07:02:00.204 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-27T07:02:00.242 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-27T07:02:00.347 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-27T07:02:00.351 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-27T07:02:00.477 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-27T07:02:00.657 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-27T07:02:00.778 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-27T07:02:01.017 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-27T07:02:01.163 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-27T07:02:01.277 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-27T07:02:01.423 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-27T07:02:01.553 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-27T07:02:01.813 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-27T07:02:01.832 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-27T07:02:02.098 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-27T07:02:02.100 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-27T07:02:02.357 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-27T07:02:02.384 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-27T07:02:02.616 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-27T07:02:02.744 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-27T07:02:02.876 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-ceph-argparse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:02.995 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-27T07:02:03.144 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-27T07:02:03.154 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:03.269 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-27T07:02:03.395 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-27T07:02:03.481 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-idna (3.3-1) ... 2024-04-27T07:02:03.665 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-27T07:02:03.789 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-27T07:02:03.824 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-27T07:02:04.049 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-27T07:02:04.418 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-27T07:02:04.455 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-27T07:02:04.518 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-27T07:02:04.619 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-27T07:02:04.979 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-27T07:02:04.988 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-27T07:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-27T07:02:05.256 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:02:05.381 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-27T07:02:05.382 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-27T07:02:05.625 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-27T07:02:05.641 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-27T07:02:05.726 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-27T07:02:05.852 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-27T07:02:05.925 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up zip (3.0-12build2) ... 2024-04-27T07:02:06.076 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-27T07:02:06.195 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-27T07:02:06.454 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-27T07:02:06.714 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:02:06.745 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-27T07:02:06.840 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-27T07:02:07.003 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-27T07:02:07.118 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-27T07:02:07.123 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-27T07:02:07.248 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-27T07:02:07.351 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-27T07:02:07.549 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-27T07:02:07.610 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-27T07:02:07.870 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-27T07:02:07.950 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-27T07:02:08.129 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-27T07:02:08.209 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-27T07:02:08.446 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-27T07:02:08.581 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-27T07:02:08.603 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-27T07:02:08.882 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-27T07:02:08.887 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-27T07:02:08.983 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-27T07:02:09.196 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-27T07:02:09.292 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-27T07:02:09.565 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-27T07:02:09.619 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-27T07:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-27T07:02:09.887 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-27T07:02:09.934 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:10.012 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-27T07:02:10.138 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-27T07:02:10.228 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-27T07:02:10.243 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-idna (3.3-1) ... 2024-04-27T07:02:10.498 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-27T07:02:10.561 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-27T07:02:10.599 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-27T07:02:10.992 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-27T07:02:11.260 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-27T07:02:11.386 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-27T07:02:11.645 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:02:11.708 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-27T07:02:11.780 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-27T07:02:11.969 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:02:12.102 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-27T07:02:12.114 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-27T07:02:12.362 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-27T07:02:12.424 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-27T07:02:12.488 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-27T07:02:12.614 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-27T07:02:12.675 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-27T07:02:12.869 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-27T07:02:12.974 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-27T07:02:13.145 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-27T07:02:13.225 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-27T07:02:13.422 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-27T07:02:13.476 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:02:13.593 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-27T07:02:13.707 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-27T07:02:13.878 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-27T07:02:14.075 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-27T07:02:14.129 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-27T07:02:14.376 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-27T07:02:14.414 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-27T07:02:14.665 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-27T07:02:14.720 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-27T07:02:14.847 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librados2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:14.916 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-27T07:02:14.964 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:15.065 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-27T07:02:15.175 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-27T07:02:15.301 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-27T07:02:15.368 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:15.494 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libradosstriper1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:15.577 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-27T07:02:15.620 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-27T07:02:15.678 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-27T07:02:15.914 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librbd1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:15.963 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-27T07:02:16.031 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-modules-core (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:16.132 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:16.272 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-27T07:02:16.339 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-27T07:02:16.339 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-27T07:02:16.523 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-27T07:02:16.625 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-27T07:02:16.842 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-27T07:02:16.842 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs-dev (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:16.968 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rados (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:17.094 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librgw2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:17.165 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-27T07:02:17.220 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephfs-top (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:17.346 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-27T07:02:17.402 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-27T07:02:17.503 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-27T07:02:17.896 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-27T07:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-27T07:02:18.315 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-27T07:02:18.574 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-27T07:02:18.700 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-27T07:02:18.783 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rbd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:18.909 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up rbd-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:19.026 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-27T07:02:19.035 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:19.161 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cephfs (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:19.287 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephfs-shell (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:19.328 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-27T07:02:19.421 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:19.587 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-27T07:02:19.790 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-27T07:02:19.888 INFO:teuthology.orchestra.run.smithi012.stdout:Adding group ceph....done 2024-04-27T07:02:20.058 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-27T07:02:20.265 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user ceph....done 2024-04-27T07:02:20.326 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-27T07:02:20.417 INFO:teuthology.orchestra.run.smithi012.stdout:Setting system user ceph properties....done 2024-04-27T07:02:20.435 INFO:teuthology.orchestra.run.smithi012.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-27T07:02:20.577 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-27T07:02:20.594 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-27T07:02:20.904 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-27T07:02:20.954 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-27T07:02:21.256 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-27T07:02:21.389 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephfs-mirror (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:21.583 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-27T07:02:21.584 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-27T07:02:21.599 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-27T07:02:21.726 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librados2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:21.953 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-27T07:02:22.040 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-test (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:22.141 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up radosgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:22.256 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libcephfs2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:22.415 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libradosstriper1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-27T07:02:22.628 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-27T07:02:22.628 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-27T07:02:22.826 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librbd1 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:22.952 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-modules-core (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:23.053 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:23.126 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-base (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:23.254 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-27T07:02:23.254 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-27T07:02:23.534 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-27T07:02:23.704 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libcephfs-dev (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:23.830 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rados (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:23.965 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up librgw2 (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:24.088 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mds (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:24.090 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up cephfs-top (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:24.217 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-27T07:02:24.335 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-27T07:02:24.335 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-27T07:02:24.831 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:25.034 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-27T07:02:25.034 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-27T07:02:25.487 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rbd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:25.507 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-osd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:25.588 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up rbd-fuse (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:25.714 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-rgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:25.774 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-27T07:02:25.774 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-27T07:02:25.840 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up python3-cephfs (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:25.966 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up cephfs-shell (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:26.092 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-common (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:26.309 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:26.435 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:26.585 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mon (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:26.601 INFO:teuthology.orchestra.run.smithi053.stdout:Adding group ceph....done 2024-04-27T07:02:26.826 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-27T07:02:26.826 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-27T07:02:26.987 INFO:teuthology.orchestra.run.smithi053.stdout:Adding system user ceph....done 2024-04-27T07:02:27.196 INFO:teuthology.orchestra.run.smithi053.stdout:Setting system user ceph properties....done 2024-04-27T07:02:27.210 INFO:teuthology.orchestra.run.smithi053.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-27T07:02:27.355 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:27.362 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-27T07:02:27.496 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:27.597 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-dashboard (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:27.687 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-27T07:02:27.723 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-volume (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:28.077 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-27T07:02:28.177 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up cephfs-mirror (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:28.185 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-27T07:02:28.364 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-27T07:02:28.408 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-27T07:02:28.408 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-27T07:02:28.869 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-test (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:28.970 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up radosgw (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:29.452 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-27T07:02:29.452 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-27T07:02:29.569 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-27T07:02:29.897 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-base (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:30.255 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-27T07:02:30.264 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-27T07:02:30.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-27T07:02:30.330 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-27T07:02:30.536 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-04-27T07:02:30.537 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-04-27T07:02:30.716 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mds (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:30.739 INFO:teuthology.orchestra.run.smithi012.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-27T07:02:30.739 INFO:teuthology.orchestra.run.smithi012.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-27T07:02:30.739 INFO:teuthology.orchestra.run.smithi012.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-27T07:02:30.739 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:02:30.739 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:02:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:02:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:02:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T07:02:30.740 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:02:30.740 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:02:30.741 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-04-27T07:02:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-27T07:02:30.741 INFO:teuthology.orchestra.run.smithi012.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-27T07:02:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-27T07:02:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-27T07:02:30.742 INFO:teuthology.orchestra.run.smithi012.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-27T07:02:30.743 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-04-27T07:02:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-27T07:02:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-27T07:02:30.743 INFO:teuthology.orchestra.run.smithi012.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-27T07:02:30.787 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-04-27T07:02:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-27T07:02:30.787 INFO:teuthology.orchestra.run.smithi012.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-27T07:02:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-27T07:02:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-27T07:02:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-27T07:02:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-27T07:02:30.788 INFO:teuthology.orchestra.run.smithi012.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-27T07:02:30.789 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2024-04-27T07:02:30.790 INFO:teuthology.orchestra.run.smithi012.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-27T07:02:30.950 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-27T07:02:30.951 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-27T07:02:30.973 INFO:teuthology.orchestra.run.smithi012.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-27T07:02:30.973 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 55.7 MB of archives. 2024-04-27T07:02:30.973 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-27T07:02:30.973 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-27T07:02:31.435 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:31.604 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-27T07:02:31.620 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-27T07:02:31.673 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-27T07:02:31.673 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-27T07:02:31.745 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-27T07:02:31.746 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-27T07:02:31.748 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-27T07:02:31.776 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-27T07:02:31.778 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-27T07:02:31.782 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-27T07:02:31.784 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-27T07:02:31.785 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-27T07:02:31.953 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-27T07:02:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-osd (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:32.246 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-27T07:02:32.255 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-27T07:02:32.260 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-27T07:02:32.346 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-27T07:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-27T07:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-27T07:02:33.132 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:33.259 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:33.410 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mon (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:33.465 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-27T07:02:33.469 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-27T07:02:33.470 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-27T07:02:33.476 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-27T07:02:33.477 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-27T07:02:33.488 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-27T07:02:33.650 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-27T07:02:33.650 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-27T07:02:33.656 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-27T07:02:34.178 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-cephadm (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:34.251 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-27T07:02:34.251 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-27T07:02:34.304 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:34.405 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-mgr-dashboard (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:34.430 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-27T07:02:34.430 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-27T07:02:34.469 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-27T07:02:34.531 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-27T07:02:34.537 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-27T07:02:34.538 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-27T07:02:34.539 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-27T07:02:34.540 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-27T07:02:34.564 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ceph-volume (19.0.0-3227-gc37f63b3-1jammy) ... 2024-04-27T07:02:34.753 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-27T07:02:34.959 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-27T07:02:35.068 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-27T07:02:35.162 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-27T07:02:35.220 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-27T07:02:35.986 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-27T07:02:35.987 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-27T07:02:35.987 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-27T07:02:35.987 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-27T07:02:35.987 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-27T07:02:35.988 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-27T07:02:36.014 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-27T07:02:36.015 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-27T07:02:36.263 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-27T07:02:36.439 INFO:teuthology.orchestra.run.smithi012.stdout:Preconfiguring packages ... 2024-04-27T07:02:36.623 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 55.7 MB in 5s (10.7 MB/s) 2024-04-27T07:02:36.885 INFO:teuthology.orchestra.run.smithi053.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-27T07:02:36.888 DEBUG:teuthology.orchestra.run.smithi053:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-27T07:02:36.940 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-27T07:02:36.946 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:02:36.952 INFO:teuthology.orchestra.run.smithi053.stdout:Reading package lists... 2024-04-27T07:02:37.099 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout:Building dependency tree... 2024-04-27T07:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout:Reading state information... 2024-04-27T07:02:37.360 INFO:teuthology.orchestra.run.smithi053.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-27T07:02:37.360 INFO:teuthology.orchestra.run.smithi053.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-27T07:02:37.360 INFO:teuthology.orchestra.run.smithi053.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-27T07:02:37.360 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages were automatically installed and are no longer required: 2024-04-27T07:02:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-27T07:02:37.360 INFO:teuthology.orchestra.run.smithi053.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-27T07:02:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-27T07:02:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-27T07:02:37.361 INFO:teuthology.orchestra.run.smithi053.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-27T07:02:37.361 INFO:teuthology.orchestra.run.smithi053.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-27T07:02:37.362 INFO:teuthology.orchestra.run.smithi053.stdout:The following additional packages will be installed: 2024-04-27T07:02:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-27T07:02:37.362 INFO:teuthology.orchestra.run.smithi053.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-27T07:02:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-27T07:02:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-27T07:02:37.363 INFO:teuthology.orchestra.run.smithi053.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-27T07:02:37.364 INFO:teuthology.orchestra.run.smithi053.stdout:Suggested packages: 2024-04-27T07:02:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-27T07:02:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-27T07:02:37.364 INFO:teuthology.orchestra.run.smithi053.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-27T07:02:37.407 INFO:teuthology.orchestra.run.smithi053.stdout:The following NEW packages will be installed: 2024-04-27T07:02:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-27T07:02:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-27T07:02:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-27T07:02:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-27T07:02:37.408 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-27T07:02:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-27T07:02:37.409 INFO:teuthology.orchestra.run.smithi053.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-27T07:02:37.410 INFO:teuthology.orchestra.run.smithi053.stdout:The following packages will be upgraded: 2024-04-27T07:02:37.410 INFO:teuthology.orchestra.run.smithi053.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-27T07:02:37.475 INFO:teuthology.orchestra.run.smithi053.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-27T07:02:37.475 INFO:teuthology.orchestra.run.smithi053.stdout:Need to get 55.7 MB of archives. 2024-04-27T07:02:37.475 INFO:teuthology.orchestra.run.smithi053.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-27T07:02:37.475 INFO:teuthology.orchestra.run.smithi053.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-27T07:02:37.636 INFO:teuthology.orchestra.run.smithi053.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-27T07:02:37.643 INFO:teuthology.orchestra.run.smithi053.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-27T07:02:37.686 INFO:teuthology.orchestra.run.smithi053.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-27T07:02:37.686 INFO:teuthology.orchestra.run.smithi053.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-27T07:02:37.687 INFO:teuthology.orchestra.run.smithi053.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-27T07:02:37.698 INFO:teuthology.orchestra.run.smithi053.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-27T07:02:37.699 INFO:teuthology.orchestra.run.smithi053.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-27T07:02:37.701 INFO:teuthology.orchestra.run.smithi053.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-27T07:02:37.703 INFO:teuthology.orchestra.run.smithi053.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-27T07:02:37.703 INFO:teuthology.orchestra.run.smithi053.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-27T07:02:37.705 INFO:teuthology.orchestra.run.smithi053.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-27T07:02:37.707 INFO:teuthology.orchestra.run.smithi053.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-27T07:02:37.727 INFO:teuthology.orchestra.run.smithi053.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-27T07:02:37.728 INFO:teuthology.orchestra.run.smithi053.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-27T07:02:37.729 INFO:teuthology.orchestra.run.smithi053.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-27T07:02:37.812 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-27T07:02:38.175 INFO:teuthology.orchestra.run.smithi053.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-27T07:02:38.178 INFO:teuthology.orchestra.run.smithi053.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-27T07:02:38.178 INFO:teuthology.orchestra.run.smithi053.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-27T07:02:38.183 INFO:teuthology.orchestra.run.smithi053.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-27T07:02:38.183 INFO:teuthology.orchestra.run.smithi053.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-27T07:02:38.190 INFO:teuthology.orchestra.run.smithi053.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-27T07:02:38.193 INFO:teuthology.orchestra.run.smithi053.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-27T07:02:38.205 INFO:teuthology.orchestra.run.smithi053.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-27T07:02:38.205 INFO:teuthology.orchestra.run.smithi053.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-27T07:02:38.238 INFO:teuthology.orchestra.run.smithi053.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-27T07:02:38.238 INFO:teuthology.orchestra.run.smithi053.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-27T07:02:38.269 INFO:teuthology.orchestra.run.smithi053.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-27T07:02:38.343 INFO:teuthology.orchestra.run.smithi053.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-27T07:02:38.349 INFO:teuthology.orchestra.run.smithi053.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-27T07:02:38.349 INFO:teuthology.orchestra.run.smithi053.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-27T07:02:38.350 INFO:teuthology.orchestra.run.smithi053.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-27T07:02:38.352 INFO:teuthology.orchestra.run.smithi053.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-27T07:02:38.378 INFO:teuthology.orchestra.run.smithi053.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-27T07:02:38.381 INFO:teuthology.orchestra.run.smithi053.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-27T07:02:38.713 INFO:teuthology.orchestra.run.smithi053.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-27T07:02:38.713 INFO:teuthology.orchestra.run.smithi053.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-27T07:02:38.713 INFO:teuthology.orchestra.run.smithi053.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-27T07:02:38.713 INFO:teuthology.orchestra.run.smithi053.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-27T07:02:38.713 INFO:teuthology.orchestra.run.smithi053.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-27T07:02:38.713 INFO:teuthology.orchestra.run.smithi053.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-27T07:02:38.729 INFO:teuthology.orchestra.run.smithi053.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-27T07:02:38.729 INFO:teuthology.orchestra.run.smithi053.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-27T07:02:38.811 INFO:teuthology.orchestra.run.smithi012.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-27T07:02:39.114 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package flex. 2024-04-27T07:02:39.153 INFO:teuthology.orchestra.run.smithi053.stdout:Preconfiguring packages ... 2024-04-27T07:02:39.218 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-27T07:02:39.223 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-27T07:02:39.270 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-27T07:02:39.315 INFO:teuthology.orchestra.run.smithi053.stdout:Fetched 55.7 MB in 1s (42.5 MB/s) 2024-04-27T07:02:39.620 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-27T07:02:39.626 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:02:39.771 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:02:40.081 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-27T07:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-27T07:02:40.386 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-27T07:02:41.073 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:02:41.326 INFO:teuthology.orchestra.run.smithi053.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-27T07:02:41.582 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package flex. 2024-04-27T07:02:41.604 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-27T07:02:41.608 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:02:41.686 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-27T07:02:41.692 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-27T07:02:41.720 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:02:41.730 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-27T07:02:42.147 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:02:42.257 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-27T07:02:42.277 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:02:42.430 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-27T07:02:42.605 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-27T07:02:42.825 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:02:42.970 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package bison. 2024-04-27T07:02:43.074 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-27T07:02:43.080 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-27T07:02:43.130 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-27T07:02:43.369 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-27T07:02:43.375 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:02:43.496 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:02:43.506 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-27T07:02:43.535 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-27T07:02:43.598 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-27T07:02:43.872 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-27T07:02:43.909 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package dump. 2024-04-27T07:02:43.938 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-27T07:02:43.968 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking dump (0.4b47-3) ... 2024-04-27T07:02:44.003 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-27T07:02:44.247 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package indent. 2024-04-27T07:02:44.275 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:44.313 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-27T07:02:44.331 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-27T07:02:44.583 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-27T07:02:44.600 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-27T07:02:44.641 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-27T07:02:44.679 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package bison. 2024-04-27T07:02:44.779 INFO:teuthology.orchestra.run.smithi053.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-27T07:02:44.784 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-27T07:02:44.832 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-27T07:02:44.936 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-27T07:02:44.953 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-27T07:02:44.986 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-27T07:02:45.165 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-27T07:02:45.194 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-27T07:02:45.223 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-27T07:02:45.232 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-27T07:02:45.240 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-27T07:02:45.282 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-27T07:02:45.577 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package dump. 2024-04-27T07:02:45.605 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-27T07:02:45.626 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-27T07:02:45.644 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking dump (0.4b47-3) ... 2024-04-27T07:02:45.655 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-27T07:02:45.693 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-27T07:02:45.937 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-27T07:02:45.939 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package indent. 2024-04-27T07:02:45.954 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-27T07:02:45.956 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:45.995 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-27T07:02:45.997 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-27T07:02:46.224 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjson-perl. 2024-04-27T07:02:46.234 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-27T07:02:46.240 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-27T07:02:46.250 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-27T07:02:46.282 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-27T07:02:46.292 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-27T07:02:46.610 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-27T07:02:46.621 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-27T07:02:46.639 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-27T07:02:46.649 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-27T07:02:46.677 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-27T07:02:46.688 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-27T07:02:46.966 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-27T07:02:46.994 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-27T07:02:47.032 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-27T07:02:47.361 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-27T07:02:47.389 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-27T07:02:47.428 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-27T07:02:47.680 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-27T07:02:47.697 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-27T07:02:47.739 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-27T07:02:47.983 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libjson-perl. 2024-04-27T07:02:48.000 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-27T07:02:48.041 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-27T07:02:48.337 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-27T07:02:48.366 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-27T07:02:48.381 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-27T07:02:48.395 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-27T07:02:48.406 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-27T07:02:48.439 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:02:48.684 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-27T07:02:48.701 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-27T07:02:48.742 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:02:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-27T07:02:49.099 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-27T07:02:49.142 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:02:49.456 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-27T07:02:49.474 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-27T07:02:49.515 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-27T07:02:49.801 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-27T07:02:49.830 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-27T07:02:49.867 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-27T07:02:50.073 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-27T07:02:50.101 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-27T07:02:50.139 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:02:50.171 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-27T07:02:50.200 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:50.229 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:02:50.411 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-27T07:02:50.437 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-27T07:02:50.484 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:02:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-27T07:02:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:50.641 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:02:50.812 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-27T07:02:50.841 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-27T07:02:50.870 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:02:50.943 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-27T07:02:50.959 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:51.001 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-27T07:02:51.220 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-27T07:02:51.235 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-27T07:02:51.236 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-27T07:02:51.273 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-27T07:02:51.295 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:02:51.610 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-27T07:02:51.638 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-27T07:02:51.677 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-27T07:02:51.791 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-27T07:02:51.819 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-27T07:02:51.849 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-27T07:02:52.029 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-27T07:02:52.058 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:52.087 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:02:52.416 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-27T07:02:52.444 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:52.482 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:02:52.728 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package modemmanager. 2024-04-27T07:02:52.757 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-27T07:02:52.795 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:02:52.802 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-27T07:02:52.830 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:52.869 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:02:53.113 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-27T07:02:53.129 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-27T07:02:53.171 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:02:53.452 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package network-manager. 2024-04-27T07:02:53.480 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-27T07:02:53.527 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-27T07:02:53.725 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-27T07:02:53.753 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-27T07:02:53.800 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-27T07:02:54.153 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package modemmanager. 2024-04-27T07:02:54.182 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-27T07:02:54.220 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:02:54.260 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ppp. 2024-04-27T07:02:54.288 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-27T07:02:54.559 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-27T07:02:54.827 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package network-manager. 2024-04-27T07:02:54.855 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-27T07:02:54.882 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package pptp-linux. 2024-04-27T07:02:54.903 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-27T07:02:54.910 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-27T07:02:54.948 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-27T07:02:55.226 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-27T07:02:55.242 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-27T07:02:55.284 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-27T07:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package ppp. 2024-04-27T07:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-27T07:02:55.587 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-27T07:02:55.616 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-27T07:02:55.653 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-client-common (238) ... 2024-04-27T07:02:55.867 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-27T07:02:55.924 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-27T07:02:55.953 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:55.991 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:02:56.173 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package pptp-linux. 2024-04-27T07:02:56.201 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-27T07:02:56.240 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-27T07:02:56.476 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-27T07:02:56.493 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-27T07:02:56.534 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-27T07:02:56.587 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-common. 2024-04-27T07:02:56.616 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-27T07:02:56.673 INFO:teuthology.orchestra.run.smithi012.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-27T07:02:56.702 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-common (238) ... 2024-04-27T07:02:56.829 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-27T07:02:56.858 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-27T07:02:56.887 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking postgresql-client-common (238) ... 2024-04-27T07:02:57.128 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-14. 2024-04-27T07:02:57.157 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:57.191 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-27T07:02:57.197 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:02:57.220 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:57.258 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:02:57.822 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package postgresql-common. 2024-04-27T07:02:57.850 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-27T07:02:57.906 INFO:teuthology.orchestra.run.smithi053.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-27T07:02:57.937 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking postgresql-common (238) ... 2024-04-27T07:02:58.379 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package postgresql-14. 2024-04-27T07:02:58.408 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-27T07:02:58.449 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:02:58.465 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql. 2024-04-27T07:02:58.495 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-27T07:02:58.540 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql (14+238) ... 2024-04-27T07:02:58.834 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-client. 2024-04-27T07:02:58.862 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-27T07:02:58.908 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-client (14+238) ... 2024-04-27T07:02:59.202 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-27T07:02:59.220 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-27T07:02:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-27T07:02:59.589 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package tcl8.6. 2024-04-27T07:02:59.617 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-27T07:02:59.655 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-27T07:02:59.875 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package postgresql. 2024-04-27T07:02:59.904 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-27T07:02:59.941 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking postgresql (14+238) ... 2024-04-27T07:02:59.975 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package tcl. 2024-04-27T07:03:00.003 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-27T07:03:00.054 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-27T07:03:00.185 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package postgresql-client. 2024-04-27T07:03:00.202 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-27T07:03:00.252 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking postgresql-client (14+238) ... 2024-04-27T07:03:00.479 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-27T07:03:00.486 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package wpasupplicant. 2024-04-27T07:03:00.508 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-27T07:03:00.514 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-27T07:03:00.546 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-27T07:03:00.552 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-27T07:03:00.899 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package tcl8.6. 2024-04-27T07:03:00.899 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-27T07:03:00.917 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-27T07:03:00.927 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-27T07:03:00.957 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-27T07:03:00.965 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-27T07:03:01.243 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package tcl. 2024-04-27T07:03:01.271 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-27T07:03:01.314 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-27T07:03:01.402 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-27T07:03:01.430 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-27T07:03:01.485 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-27T07:03:01.663 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package wpasupplicant. 2024-04-27T07:03:01.691 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-27T07:03:01.729 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-27T07:03:01.781 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-client-common (238) ... 2024-04-27T07:03:01.991 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-27T07:03:02.075 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-27T07:03:02.092 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-27T07:03:02.117 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-27T07:03:02.133 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-27T07:03:02.243 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:03:02.361 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-27T07:03:02.462 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-27T07:03:02.470 INFO:teuthology.orchestra.run.smithi053.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-27T07:03:02.499 INFO:teuthology.orchestra.run.smithi053.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-27T07:03:02.545 INFO:teuthology.orchestra.run.smithi053.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-27T07:03:02.580 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:03:02.825 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up postgresql-client-common (238) ... 2024-04-27T07:03:03.035 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-27T07:03:03.145 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-27T07:03:03.169 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-27T07:03:03.228 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-27T07:03:03.295 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:03:03.446 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-27T07:03:03.613 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-27T07:03:03.621 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-27T07:03:03.621 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-27T07:03:04.149 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:03:04.164 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-27T07:03:04.290 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-27T07:03:04.391 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:03:04.492 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-27T07:03:04.609 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-27T07:03:04.717 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-27T07:03:04.735 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-27T07:03:04.805 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-27T07:03:04.862 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-27T07:03:04.987 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-27T07:03:05.182 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-27T07:03:05.182 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-27T07:03:05.626 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-27T07:03:05.699 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-27T07:03:05.777 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up dump (0.4b47-3) ... 2024-04-27T07:03:05.826 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-27T07:03:05.888 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-27T07:03:05.952 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-27T07:03:05.963 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:03:06.078 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-27T07:03:06.105 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-27T07:03:06.204 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-27T07:03:06.231 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-27T07:03:06.330 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-27T07:03:06.357 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-27T07:03:06.456 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-27T07:03:06.484 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-27T07:03:06.574 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-27T07:03:06.582 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-27T07:03:06.677 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-27T07:03:06.828 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-27T07:03:07.037 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-27T07:03:07.172 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:03:07.205 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-27T07:03:07.298 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-27T07:03:07.306 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up dump (0.4b47-3) ... 2024-04-27T07:03:07.399 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-27T07:03:07.466 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-client (14+238) ... 2024-04-27T07:03:07.482 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:03:07.592 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-27T07:03:07.608 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-27T07:03:07.734 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-27T07:03:07.860 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-27T07:03:07.986 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-27T07:03:08.059 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-27T07:03:08.109 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:08.109 INFO:teuthology.orchestra.run.smithi012.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-27T07:03:08.109 INFO:teuthology.orchestra.run.smithi012.stdout:which means they are currently configured by ifupdown: 2024-04-27T07:03:08.109 INFO:teuthology.orchestra.run.smithi012.stdout:- ens1f0 2024-04-27T07:03:08.109 INFO:teuthology.orchestra.run.smithi012.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-27T07:03:08.109 INFO:teuthology.orchestra.run.smithi012.stdout:remove their configuration from /etc/network/interfaces. 2024-04-27T07:03:08.109 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:08.130 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-27T07:03:08.231 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-27T07:03:08.242 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-27T07:03:08.407 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-27T07:03:08.550 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:03:08.572 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-27T07:03:08.676 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-27T07:03:08.844 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up postgresql-client (14+238) ... 2024-04-27T07:03:08.900 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-27T07:03:08.970 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-27T07:03:09.403 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:03:09.525 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:09.525 INFO:teuthology.orchestra.run.smithi053.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-27T07:03:09.526 INFO:teuthology.orchestra.run.smithi053.stdout:which means they are currently configured by ifupdown: 2024-04-27T07:03:09.526 INFO:teuthology.orchestra.run.smithi053.stdout:- ens1f0 2024-04-27T07:03:09.526 INFO:teuthology.orchestra.run.smithi053.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-27T07:03:09.526 INFO:teuthology.orchestra.run.smithi053.stdout:remove their configuration from /etc/network/interfaces. 2024-04-27T07:03:09.526 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:09.565 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-27T07:03:09.658 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-27T07:03:09.691 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-27T07:03:09.817 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-common (238) ... 2024-04-27T07:03:09.967 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-27T07:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-27T07:03:10.780 INFO:teuthology.orchestra.run.smithi012.stdout:Adding user postgres to group ssl-cert 2024-04-27T07:03:10.826 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-27T07:03:11.053 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:11.053 INFO:teuthology.orchestra.run.smithi012.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-27T07:03:11.054 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-27T07:03:11.240 INFO:teuthology.orchestra.run.smithi012.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-27T07:03:11.240 INFO:teuthology.orchestra.run.smithi012.stdout:Removing obsolete dictionary files: 2024-04-27T07:03:11.306 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-27T07:03:11.481 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up postgresql-common (238) ... 2024-04-27T07:03:11.807 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-27T07:03:12.202 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:03:12.561 INFO:teuthology.orchestra.run.smithi012.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-27T07:03:12.575 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-27T07:03:12.579 INFO:teuthology.orchestra.run.smithi053.stdout:Adding user postgres to group ssl-cert 2024-04-27T07:03:12.617 INFO:teuthology.orchestra.run.smithi012.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-27T07:03:12.617 INFO:teuthology.orchestra.run.smithi012.stdout:This user must also own the server process. 2024-04-27T07:03:12.618 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:12.618 INFO:teuthology.orchestra.run.smithi012.stdout:The database cluster will be initialized with locale "en_US". 2024-04-27T07:03:12.618 INFO:teuthology.orchestra.run.smithi012.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-27T07:03:12.618 INFO:teuthology.orchestra.run.smithi012.stdout:The default text search configuration will be set to "english". 2024-04-27T07:03:12.618 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:12.618 INFO:teuthology.orchestra.run.smithi012.stdout:Data page checksums are disabled. 2024-04-27T07:03:12.618 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:12.618 INFO:teuthology.orchestra.run.smithi012.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-27T07:03:12.618 INFO:teuthology.orchestra.run.smithi012.stdout:creating subdirectories ... ok 2024-04-27T07:03:12.618 INFO:teuthology.orchestra.run.smithi012.stdout:selecting dynamic shared memory implementation ... posix 2024-04-27T07:03:12.634 INFO:teuthology.orchestra.run.smithi012.stdout:selecting default max_connections ... 100 2024-04-27T07:03:12.655 INFO:teuthology.orchestra.run.smithi012.stdout:selecting default shared_buffers ... 128MB 2024-04-27T07:03:12.662 INFO:teuthology.orchestra.run.smithi012.stdout:selecting default time zone ... Etc/UTC 2024-04-27T07:03:12.663 INFO:teuthology.orchestra.run.smithi012.stdout:creating configuration files ... ok 2024-04-27T07:03:12.778 INFO:teuthology.orchestra.run.smithi012.stdout:running bootstrap script ... ok 2024-04-27T07:03:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:12.816 INFO:teuthology.orchestra.run.smithi053.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-27T07:03:12.993 INFO:teuthology.orchestra.run.smithi053.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-27T07:03:12.993 INFO:teuthology.orchestra.run.smithi053.stdout:Removing obsolete dictionary files: 2024-04-27T07:03:13.433 INFO:teuthology.orchestra.run.smithi012.stdout:performing post-bootstrap initialization ... ok 2024-04-27T07:03:13.544 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-27T07:03:13.947 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-27T07:03:14.095 INFO:teuthology.orchestra.run.smithi012.stdout:syncing data to disk ... ok 2024-04-27T07:03:14.289 INFO:teuthology.orchestra.run.smithi053.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-27T07:03:14.304 INFO:teuthology.orchestra.run.smithi053.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-27T07:03:14.357 INFO:teuthology.orchestra.run.smithi053.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-27T07:03:14.357 INFO:teuthology.orchestra.run.smithi053.stdout:This user must also own the server process. 2024-04-27T07:03:14.357 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:14.357 INFO:teuthology.orchestra.run.smithi053.stdout:The database cluster will be initialized with locale "en_US". 2024-04-27T07:03:14.357 INFO:teuthology.orchestra.run.smithi053.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-27T07:03:14.358 INFO:teuthology.orchestra.run.smithi053.stdout:The default text search configuration will be set to "english". 2024-04-27T07:03:14.358 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:14.358 INFO:teuthology.orchestra.run.smithi053.stdout:Data page checksums are disabled. 2024-04-27T07:03:14.358 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:14.358 INFO:teuthology.orchestra.run.smithi053.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-27T07:03:14.358 INFO:teuthology.orchestra.run.smithi053.stdout:creating subdirectories ... ok 2024-04-27T07:03:14.358 INFO:teuthology.orchestra.run.smithi053.stdout:selecting dynamic shared memory implementation ... posix 2024-04-27T07:03:14.368 INFO:teuthology.orchestra.run.smithi053.stdout:selecting default max_connections ... 100 2024-04-27T07:03:14.386 INFO:teuthology.orchestra.run.smithi053.stdout:selecting default shared_buffers ... 128MB 2024-04-27T07:03:14.396 INFO:teuthology.orchestra.run.smithi053.stdout:selecting default time zone ... Etc/UTC 2024-04-27T07:03:14.396 INFO:teuthology.orchestra.run.smithi053.stdout:creating configuration files ... ok 2024-04-27T07:03:14.504 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-27T07:03:14.515 INFO:teuthology.orchestra.run.smithi053.stdout:running bootstrap script ... ok 2024-04-27T07:03:15.153 INFO:teuthology.orchestra.run.smithi053.stdout:performing post-bootstrap initialization ... ok 2024-04-27T07:03:15.833 INFO:teuthology.orchestra.run.smithi053.stdout:syncing data to disk ... ok 2024-04-27T07:03:16.225 INFO:teuthology.orchestra.run.smithi053.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-27T07:03:17.266 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:03:17.428 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-27T07:03:17.568 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql (14+238) ... 2024-04-27T07:03:17.719 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:03:17.871 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:03:18.131 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-27T07:03:18.131 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-27T07:03:18.665 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-27T07:03:18.766 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-27T07:03:18.973 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:03:19.059 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-27T07:03:19.081 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-27T07:03:19.207 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up postgresql (14+238) ... 2024-04-27T07:03:19.230 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-27T07:03:19.332 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-27T07:03:19.459 INFO:teuthology.orchestra.run.smithi053.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-27T07:03:19.720 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-27T07:03:19.720 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-27T07:03:20.270 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-27T07:03:20.370 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-27T07:03:20.631 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-27T07:03:20.656 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-27T07:03:20.731 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-27T07:03:20.792 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-27T07:03:21.275 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-27T07:03:21.281 DEBUG:teuthology.parallel:result is None 2024-04-27T07:03:21.878 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-27T07:03:21.978 INFO:teuthology.orchestra.run.smithi053.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-27T07:03:22.452 INFO:teuthology.orchestra.run.smithi053.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-27T07:03:22.457 DEBUG:teuthology.parallel:result is None 2024-04-27T07:03:22.458 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-27T07:03:22.691 DEBUG:teuthology.orchestra.run.smithi012:> dpkg-query -W -f '${Version}' ceph 2024-04-27T07:03:22.711 INFO:teuthology.orchestra.run.smithi012.stdout:19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:03:22.711 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:03:22.711 INFO:teuthology.task.install:The correct ceph version 19.0.0-3227-gc37f63b3-1jammy is installed. 2024-04-27T07:03:22.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-27T07:03:22.851 DEBUG:teuthology.orchestra.run.smithi053:> dpkg-query -W -f '${Version}' ceph 2024-04-27T07:03:22.871 INFO:teuthology.orchestra.run.smithi053.stdout:19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:03:22.872 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3227-gc37f63b3-1jammy 2024-04-27T07:03:22.872 INFO:teuthology.task.install:The correct ceph version 19.0.0-3227-gc37f63b3-1jammy is installed. 2024-04-27T07:03:22.874 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-27T07:03:22.874 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:22.874 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T07:03:22.889 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:22.889 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T07:03:22.929 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-27T07:03:22.929 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:22.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T07:03:22.945 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T07:03:22.999 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:23.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T07:03:23.013 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T07:03:23.068 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-27T07:03:23.068 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:23.068 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T07:03:23.082 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T07:03:23.136 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:23.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T07:03:23.149 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T07:03:23.203 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-27T07:03:23.203 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:23.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T07:03:23.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T07:03:23.272 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:23.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T07:03:23.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T07:03:23.340 INFO:teuthology.run_tasks:Running task ceph... 2024-04-27T07:03:23.433 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-27T07:03:23.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /var/log/ceph 2024-04-27T07:03:23.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /var/log/ceph 2024-04-27T07:03:23.449 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-27T07:03:23.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-27T07:03:23.493 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-27T07:03:23.507 INFO:tasks.ceph:Creating extra log directories... 2024-04-27T07:03:23.507 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-27T07:03:23.549 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-27T07:03:23.569 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-27T07:03:23.569 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-27T07:03:23.570 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-27T07:03:23.570 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:23.570 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-27T07:03:23.609 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-27T07:03:23.685 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:23.686 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-27T07:03:23.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-27T07:03:23.774 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-27T07:03:23.774 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-27T07:03:23.774 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037', 'branch': 'main', 'description': 'fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672037', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '4e213a58e787437ff508e6c890ccd5fd32dc96b5'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 4710, 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f', 'sleep_before_teardown': 0, 'subset': '356/512', 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e213a58e787437ff508e6c890ccd5fd32dc96b5', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi012.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBByKoS5wtV294iofbbkD3c3JmT9A/H+k//+OspIbHvtlCiRYHBcAtpsx8ZGlqxT7/kBjYs8ihx6uCGZlvOLDaJU=', 'smithi053.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMRr6fEgH9O7yAPsE7IESr/UoTs+hNtQFq2EUtgh9JWnpxRplmFRVRRPDppla624f4eBnmwhreWF1I7Z9teX5fk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cephfs_shell']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_20:24:13', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-27T07:03:23.775 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-27T07:03:23.777 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-27T07:03:23.817 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/run/ceph 2024-04-27T07:03:23.825 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/run/ceph 2024-04-27T07:03:23.873 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:23.873 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T07:03:23.884 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T07:03:23.884 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-04-27T07:03:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T07:03:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:03:23.933 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 879 Links: 1 2024-04-27T07:03:23.933 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:03:23.933 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-27 06:56:46.665083763 +0000 2024-04-27T07:03:23.933 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-27 06:56:46.393086193 +0000 2024-04-27T07:03:23.933 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-27 06:56:46.393086193 +0000 2024-04-27T07:03:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-04-27T07:03:23.933 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T07:03:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-27T07:03:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-27T07:03:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000423227 s, 1.2 MB/s 2024-04-27T07:03:23.990 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T07:03:24.039 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-04-27T07:03:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T07:03:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:03:24.089 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-04-27T07:03:24.089 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:03:24.089 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-27 06:56:46.917081513 +0000 2024-04-27T07:03:24.089 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-27 06:56:46.913081548 +0000 2024-04-27T07:03:24.089 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-27 06:56:46.913081548 +0000 2024-04-27T07:03:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-04-27T07:03:24.089 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T07:03:24.144 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-27T07:03:24.145 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-27T07:03:24.145 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000324989 s, 1.6 MB/s 2024-04-27T07:03:24.146 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T07:03:24.197 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-04-27T07:03:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T07:03:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:03:24.246 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-04-27T07:03:24.246 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:03:24.246 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-27 06:56:47.641075045 +0000 2024-04-27T07:03:24.246 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-27 06:56:47.385077333 +0000 2024-04-27T07:03:24.246 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-27 06:56:47.385077333 +0000 2024-04-27T07:03:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-04-27T07:03:24.247 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T07:03:24.304 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-27T07:03:24.304 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-27T07:03:24.304 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000441177 s, 1.2 MB/s 2024-04-27T07:03:24.305 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T07:03:24.354 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-04-27T07:03:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T07:03:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:03:24.401 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 930 Links: 1 2024-04-27T07:03:24.401 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:03:24.401 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-27 06:56:47.877072937 +0000 2024-04-27T07:03:24.401 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-27 06:56:47.873072973 +0000 2024-04-27T07:03:24.402 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-27 06:56:47.873072973 +0000 2024-04-27T07:03:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-04-27T07:03:24.402 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T07:03:24.456 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-27T07:03:24.456 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-27T07:03:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000292517 s, 1.8 MB/s 2024-04-27T07:03:24.458 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T07:03:24.506 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-27T07:03:24.506 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:24.507 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T07:03:24.512 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T07:03:24.512 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-04-27T07:03:24.560 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T07:03:24.561 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:03:24.561 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-04-27T07:03:24.561 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:03:24.561 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-27 06:56:48.753018556 +0000 2024-04-27T07:03:24.561 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-27 06:56:48.485020945 +0000 2024-04-27T07:03:24.561 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-27 06:56:48.485020945 +0000 2024-04-27T07:03:24.561 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-04-27T07:03:24.561 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T07:03:24.615 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-27T07:03:24.616 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-27T07:03:24.616 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000328881 s, 1.6 MB/s 2024-04-27T07:03:24.617 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T07:03:24.665 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-04-27T07:03:24.713 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T07:03:24.713 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:03:24.713 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-04-27T07:03:24.713 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:03:24.713 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-27 06:56:49.269013960 +0000 2024-04-27T07:03:24.714 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-27 06:56:48.997016383 +0000 2024-04-27T07:03:24.714 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-27 06:56:48.997016383 +0000 2024-04-27T07:03:24.714 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-04-27T07:03:24.714 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T07:03:24.766 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-27T07:03:24.766 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-27T07:03:24.766 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000270741 s, 1.9 MB/s 2024-04-27T07:03:24.768 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T07:03:24.818 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-04-27T07:03:24.865 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T07:03:24.865 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:03:24.865 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 918 Links: 1 2024-04-27T07:03:24.865 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:03:24.865 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-27 06:56:49.781009398 +0000 2024-04-27T07:03:24.865 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-27 06:56:49.497011928 +0000 2024-04-27T07:03:24.865 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-27 06:56:49.497011928 +0000 2024-04-27T07:03:24.865 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-04-27T07:03:24.865 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T07:03:24.918 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-27T07:03:24.918 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-27T07:03:24.919 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.0002234 s, 2.3 MB/s 2024-04-27T07:03:24.920 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T07:03:24.969 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-04-27T07:03:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T07:03:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T07:03:25.017 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 931 Links: 1 2024-04-27T07:03:25.017 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T07:03:25.017 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-04-27 06:56:50.269005050 +0000 2024-04-27T07:03:25.017 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-04-27 06:56:50.005007402 +0000 2024-04-27T07:03:25.017 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-04-27 06:56:50.005007402 +0000 2024-04-27T07:03:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2024-04-27T07:03:25.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T07:03:25.071 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-04-27T07:03:25.071 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-04-27T07:03:25.071 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000308726 s, 1.7 MB/s 2024-04-27T07:03:25.072 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T07:03:25.122 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-27T07:03:25.122 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-27T07:03:25.123 INFO:tasks.ceph:Generating config... 2024-04-27T07:03:25.124 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-27T07:03:25.124 INFO:tasks.ceph:[client] debug client = 20 2024-04-27T07:03:25.124 INFO:tasks.ceph:[client] debug ms = 1 2024-04-27T07:03:25.124 INFO:tasks.ceph:[client] pid file = 2024-04-27T07:03:25.124 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-27T07:03:25.124 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-27T07:03:25.124 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-27T07:03:25.124 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-27T07:03:25.124 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-27T07:03:25.124 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-27T07:03:25.124 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-27T07:03:25.124 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-27T07:03:25.124 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-27T07:03:25.124 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-27T07:03:25.125 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-27T07:03:25.125 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-27T07:03:25.125 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-27T07:03:25.125 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-27T07:03:25.125 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-27T07:03:25.125 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-27T07:03:25.125 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-27T07:03:25.125 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-27T07:03:25.125 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-27T07:03:25.125 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-27T07:03:25.125 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-27T07:03:25.125 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-27T07:03:25.125 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-27T07:03:25.125 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-27T07:03:25.125 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-27T07:03:25.125 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-27T07:03:25.126 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-27T07:03:25.126 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-27T07:03:25.126 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-27T07:03:25.126 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-27T07:03:25.126 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-27T07:03:25.126 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-27T07:03:25.126 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-27T07:03:25.126 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-27T07:03:25.126 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-27T07:03:25.126 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-27T07:03:25.126 INFO:tasks.ceph:Setting up mon.a... 2024-04-27T07:03:25.126 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-27T07:03:25.162 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.keyring 2024-04-27T07:03:25.180 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-27T07:03:25.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T07:03:25.319 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.12'), ('mon.b', '172.21.15.53'), ('mon.c', '[v2:172.21.15.53:3301,v1:172.21.15.53:6790]')] 2024-04-27T07:03:25.320 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.12,172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-27T07:03:25.321 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:25.321 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-27T07:03:25.368 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.12 --add b 172.21.15.53 --addv c '[v2:172.21.15.53:3301,v1:172.21.15.53:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:25.458 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: generated fsid e023690c-eecf-4ce8-af7e-ab331203966f 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = quincy 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 0 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:fsid e023690c-eecf-4ce8-af7e-ab331203966f 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:last_changed 2024-04-27T07:03:25.457969+0000 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-04-27T07:03:25.457969+0000 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:min_mon_release 17 (quincy) 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:election_strategy: 1 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:2: [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] mon.c 2024-04-27T07:03:25.459 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-27T07:03:25.461 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-27T07:03:25.509 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e023690c-eecf-4ce8-af7e-ab331203966f... 2024-04-27T07:03:25.510 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-27T07:03:25.553 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-27T07:03:25.582 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-04-27T07:03:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: chdir = "" 2024-04-27T07:03:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: auth supported = cephx 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: filestore xattr use omap = true 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: mon clock drift allowed = 1.000 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: osd crush chooseleaf type = 0 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: auth debug = true 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on old message = true 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on bug = true 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: mon pg warn max object skew = 0 2024-04-27T07:03:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: # disable pg_autoscaler by default for new pools 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default size = 2 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow primary affinity = true 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow pg remap = true 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on legacy crush tunables = false 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on crush straw calc version zero = false 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on no sortbitwise = false 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on osd down out interval zero = false 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on too few osds = false 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: mon_allow_pool_size_one = true 2024-04-27T07:03:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: osd default data pool replay window = 5 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: mon allow pool delete = true 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: mon cluster log file level = debug 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: debug asserts on shutdown = true 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: mon health detail to clog = false 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: mon host = "172.21.15.12,172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790]" 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = e023690c-eecf-4ce8-af7e-ab331203966f 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout:[osd] 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: osd journal size = 100 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub load threshold = 5.0 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub max interval = 600 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock profile = high_recovery_ops 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd recover clone overlap = true 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd recovery max chunk = 1048576 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug shutdown = true 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug op order = true 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify stray on activate = true 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug trim objects = true 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd open classes on start = true 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug pg log writeout = true 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: osd deep scrub update digest min age = 30 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: osd map max advance = 10 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: journal zero on create = true 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: filestore ondisk finisher threads = 3 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: filestore apply finisher threads = 3 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: bdev debug aio = true 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug misdirected ops = true 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: bdev async discard = True 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: bdev enable discard = True 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore allocator = bitmap 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore block size = 96636764160 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore fsck on mount = True 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluefs = 1/20 2024-04-27T07:03:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluestore = 1/20 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: debug osd = 20 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: debug rocksdb = 4/10 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd backfillfull_ratio = 0.85 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd full ratio = 0.9 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd nearfull ratio = 0.8 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: osd failsafe full ratio = 0.95 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: osd objectstore = bluestore 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: osd op complaint time = 180 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr] 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: debug mgr = 20 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2024-04-27T07:03:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: mgr/telemetry/nag = false 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: debug client = 20 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout:[mon] 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: debug paxos = 20 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: mon data avail warn = 5 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: mon mgr mkfs grace = 240 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd reporter subtree level = osd 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd prime pg temp = true 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: auth mon ticket ttl = 660 # 11m 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: auth service ticket ttl = 240 # 4m 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: # don't complain about insecure global_id in the test suite 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: # 1m isn't quite enough 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: mon_down_mkfs_grace = 2m 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_filestore_osds = false 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: mon down mkfs grace = 300 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: mon op complaint time = 120 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:03:25.590 INFO:teuthology.orchestra.run.smithi012.stdout:[client] 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: rgw cache enabled = true 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable ops log = true 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable usage log = true 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: client mount timeout = 600 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: debug client = 20 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = "" 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: rados mon op timeout = 900 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: rados osd op timeout = 900 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.a] 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.b] 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.c] 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a] 2024-04-27T07:03:25.591 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b] 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout:[mds] 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: debug mds = 20 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: debug mds balancer = 20 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: mds debug frag = True 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: mds debug scatterstat = True 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: mds op complaint time = 180 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: mds verify scatter = True 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: osd op complaint time = 180 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: rados mon op timeout = 900 2024-04-27T07:03:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: rados osd op timeout = 900 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2024-04-27T07:03:25.593 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2024-04-27T07:03:25.594 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2024-04-27T07:03:25.595 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.12,172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790]" 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = e023690c-eecf-4ce8-af7e-ab331203966f 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.596 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug trim objects = true 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2024-04-27T07:03:25.597 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: bdev async discard = True 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: bdev enable discard = True 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore allocator = bitmap 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 1/20 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 1/20 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 4/10 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2024-04-27T07:03:25.598 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2024-04-27T07:03:25.599 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2024-04-27T07:03:25.600 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: mon down mkfs grace = 300 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: mon op complaint time = 120 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-27T07:03:25.601 INFO:teuthology.orchestra.run.smithi053.stdout: client mount timeout = 600 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout: debug client = 20 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = "" 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout:[mds] 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds = 20 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout: debug mds balancer = 20 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug frag = True 2024-04-27T07:03:25.602 INFO:teuthology.orchestra.run.smithi053.stdout: mds debug scatterstat = True 2024-04-27T07:03:25.603 INFO:teuthology.orchestra.run.smithi053.stdout: mds op complaint time = 180 2024-04-27T07:03:25.603 INFO:teuthology.orchestra.run.smithi053.stdout: mds verify scatter = True 2024-04-27T07:03:25.603 INFO:teuthology.orchestra.run.smithi053.stdout: osd op complaint time = 180 2024-04-27T07:03:25.603 INFO:teuthology.orchestra.run.smithi053.stdout: rados mon op timeout = 900 2024-04-27T07:03:25.603 INFO:teuthology.orchestra.run.smithi053.stdout: rados osd op timeout = 900 2024-04-27T07:03:25.604 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-27T07:03:25.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-27T07:03:25.698 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-27T07:03:25.698 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:25.698 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-27T07:03:25.703 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:25.703 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-27T07:03:25.752 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:03:25.752 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:25.752 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-27T07:03:25.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T07:03:25.822 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:25.822 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:25.888 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T07:03:25.888 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:25.888 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-27T07:03:25.888 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T07:03:25.913 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:25.913 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:25.965 INFO:tasks.ceph:Setting up mon nodes... 2024-04-27T07:03:25.965 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-27T07:03:25.965 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-27T07:03:26.011 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-27T07:03:26.014 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-27T07:03:26.059 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-27T07:03:26.072 INFO:tasks.ceph:Setting up mds nodes... 2024-04-27T07:03:26.073 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-27T07:03:26.119 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-27T07:03:26.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-27T07:03:26.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-27T07:03:26.221 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-27T07:03:26.225 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-27T07:03:26.284 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-27T07:03:26.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-27T07:03:26.321 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-27T07:03:26.332 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-27T07:03:26.333 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-27T07:03:26.333 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-27T07:03:26.388 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-27T07:03:26.388 INFO:tasks.ceph:role: osd.0 2024-04-27T07:03:26.388 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:03:26.388 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-27T07:03:26.456 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:03:26.456 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:03:26.456 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:03:26.457 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:03:26.457 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:03:26.457 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:03:26.457 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:03:26.457 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:03:26.457 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:03:26.457 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:03:26.917 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-04-27T07:03:26.919 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-04-27T07:03:26.919 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-27T07:03:27.016 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-27T07:03:27.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:03:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:03:27.027 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-27T07:03:27.079 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-27T07:03:27.079 INFO:tasks.ceph:role: osd.1 2024-04-27T07:03:27.079 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:03:27.079 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:03:27.145 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:03:27.146 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:03:27.620 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-04-27T07:03:27.622 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-04-27T07:03:27.623 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-27T07:03:27.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-27T07:03:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:03:27.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:03:27.782 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-27T07:03:27.840 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-27T07:03:27.840 INFO:tasks.ceph:role: osd.2 2024-04-27T07:03:27.840 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:03:27.840 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-27T07:03:27.903 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:03:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:03:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:03:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:03:27.903 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:03:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:03:27.903 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:03:27.903 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:03:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:03:27.904 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:03:28.381 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-04-27T07:03:28.383 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-04-27T07:03:28.383 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-27T07:03:28.527 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-27T07:03:28.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:03:28.539 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:03:28.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-27T07:03:28.592 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-27T07:03:28.592 INFO:tasks.ceph:role: osd.3 2024-04-27T07:03:28.592 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:03:28.592 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-27T07:03:28.656 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:03:28.657 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:03:28.657 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:03:28.657 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:03:28.657 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:03:28.657 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:03:28.657 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:03:28.657 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:03:28.657 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:03:28.657 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:03:29.083 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-04-27T07:03:29.085 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-04-27T07:03:29.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-27T07:03:29.186 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-27T07:03:29.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:03:29.196 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:03:29.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:29.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:29.301+0000 7fa2ac1aa7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-27T07:03:29.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:29.305+0000 7fa2ac1aa7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-27T07:03:29.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:29.305+0000 7fa2ac1aa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T07:03:29.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:29.305+0000 7fa2ac1aa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T07:03:29.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:29.305+0000 7fa2ac1aa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T07:03:29.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:29.305+0000 7fa2ac1aa7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-27T07:03:32.296 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-27T07:03:32.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:32.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:32.397+0000 7f2cb20fb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-27T07:03:32.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:32.397+0000 7f2cb20fb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-27T07:03:32.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:32.397+0000 7f2cb20fb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T07:03:32.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:32.397+0000 7f2cb20fb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T07:03:32.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:32.397+0000 7f2cb20fb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T07:03:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:32.397+0000 7f2cb20fb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-27T07:03:35.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-27T07:03:35.422 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:35.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:35.497+0000 7f39ba5a37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-27T07:03:35.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:35.501+0000 7f39ba5a37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-27T07:03:35.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:35.501+0000 7f39ba5a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T07:03:35.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:35.501+0000 7f39ba5a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T07:03:35.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:35.501+0000 7f39ba5a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T07:03:35.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:35.501+0000 7f39ba5a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-27T07:03:38.492 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-27T07:03:38.553 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:38.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:38.633+0000 7fd11cb7e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-27T07:03:38.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:38.633+0000 7fd11cb7e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-27T07:03:38.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:38.633+0000 7fd11cb7e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T07:03:38.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:38.633+0000 7fd11cb7e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T07:03:38.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:38.633+0000 7fd11cb7e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T07:03:38.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:38.633+0000 7fd11cb7e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-27T07:03:41.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-27T07:03:41.654 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-27T07:03:41.669 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-27T07:03:41.669 INFO:tasks.ceph:role: osd.4 2024-04-27T07:03:41.669 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T07:03:41.669 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-27T07:03:41.725 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:03:41.725 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:03:41.725 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:03:41.725 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:03:41.725 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:03:41.725 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:03:41.725 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:03:41.725 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:03:41.725 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:03:41.726 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:03:41.749 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-04-27T07:03:41.751 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-04-27T07:03:41.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-27T07:03:41.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-27T07:03:41.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:03:41.857 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:03:41.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-27T07:03:41.910 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-27T07:03:41.910 INFO:tasks.ceph:role: osd.5 2024-04-27T07:03:41.910 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T07:03:41.911 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-27T07:03:41.964 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:03:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:03:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:03:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:03:41.964 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:03:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:03:41.964 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:03:41.965 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:03:41.965 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:03:41.965 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:03:41.988 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-04-27T07:03:41.989 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-04-27T07:03:41.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-27T07:03:42.048 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-27T07:03:42.102 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:03:42.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:03:42.104 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-27T07:03:42.161 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-27T07:03:42.161 INFO:tasks.ceph:role: osd.6 2024-04-27T07:03:42.161 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T07:03:42.161 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-27T07:03:42.218 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:03:42.218 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:03:42.219 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:03:42.219 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:03:42.219 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:03:42.219 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:03:42.219 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:03:42.219 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:03:42.219 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:03:42.219 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:03:42.243 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-04-27T07:03:42.245 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-04-27T07:03:42.245 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-27T07:03:42.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-27T07:03:42.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:03:42.358 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:03:42.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-27T07:03:42.413 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-27T07:03:42.413 INFO:tasks.ceph:role: osd.7 2024-04-27T07:03:42.414 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T07:03:42.414 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-27T07:03:42.468 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T07:03:42.468 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T07:03:42.468 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T07:03:42.468 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-27T07:03:42.468 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T07:03:42.469 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-04-27T07:03:42.469 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T07:03:42.469 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-27T07:03:42.469 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T07:03:42.469 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T07:03:42.493 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-04-27T07:03:42.495 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-04-27T07:03:42.495 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-27T07:03:42.552 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-27T07:03:42.605 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: /sbin/restorecon: command not found 2024-04-27T07:03:42.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:03:42.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:42.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:42.712+0000 7fe1623357c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-27T07:03:42.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:42.712+0000 7fe1623357c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-27T07:03:42.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:42.712+0000 7fe1623357c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-27T07:03:42.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:42.712+0000 7fe1623357c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-27T07:03:42.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:42.712+0000 7fe1623357c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-27T07:03:42.714 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:42.712+0000 7fe1623357c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-27T07:03:45.712 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-27T07:03:45.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:45.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:45.812+0000 7f5b054c47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-27T07:03:45.814 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:45.812+0000 7f5b054c47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-27T07:03:45.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:45.812+0000 7f5b054c47c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-27T07:03:45.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:45.812+0000 7f5b054c47c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-27T07:03:45.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:45.812+0000 7f5b054c47c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-27T07:03:45.815 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:45.812+0000 7f5b054c47c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-27T07:03:48.802 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-27T07:03:48.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:48.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:48.939+0000 7f9f0a8027c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-27T07:03:48.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:48.939+0000 7f9f0a8027c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-27T07:03:48.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:48.939+0000 7f9f0a8027c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-27T07:03:48.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:48.939+0000 7f9f0a8027c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-27T07:03:48.944 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:48.939+0000 7f9f0a8027c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-27T07:03:48.945 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:48.943+0000 7f9f0a8027c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-27T07:03:51.913 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-27T07:03:51.975 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:52.059+0000 7f9f408907c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-27T07:03:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:52.059+0000 7f9f408907c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-27T07:03:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:52.059+0000 7f9f408907c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-27T07:03:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:52.059+0000 7f9f408907c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-27T07:03:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:52.059+0000 7f9f408907c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-27T07:03:52.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:52.059+0000 7f9f408907c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-27T07:03:53.777 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:03:53.780 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:03:55.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-27T07:03:55.074 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-27T07:03:55.075 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:55.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-27T07:03:55.089 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:55.089 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-27T07:03:55.145 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:55.145 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-27T07:03:55.201 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:55.202 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-27T07:03:55.256 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:55.256 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-27T07:03:55.313 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:55.313 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-27T07:03:55.368 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:55.368 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-27T07:03:55.383 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:55.383 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-27T07:03:55.438 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:55.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-27T07:03:55.497 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:55.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-27T07:03:55.553 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:55.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-27T07:03:55.613 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:55.613 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-27T07:03:55.670 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:55.670 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-27T07:03:55.676 INFO:tasks.ceph:Adding keys to all mons... 2024-04-27T07:03:55.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-27T07:03:55.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-27T07:03:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.y] 2024-04-27T07:03:55.731 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQA+oyxm+NugABAAzPlk/x6yFXaTG5J0uLdcOA== 2024-04-27T07:03:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a] 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQA+oyxmeosVBxAA64Na7IWgq/9JX8IryFKwqQ== 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.0] 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBBoyxm9cBBEhAA+7TsG1DI1x9B58lFniRTCA== 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.1] 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBEoyxmUkfXFxAAeOVSfqw19vxB9lh6A+B2mw== 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.2] 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBHoyxm05j6HRAASwIItsLH5QWhSJJua+scaQ== 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.3] 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBKoyxmIUTjJRAAXn6H3GX7kIiyF6rxixzGxg== 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.x] 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQA+oyxmryeZAxAAbc4+8Txh5jr/zMCfbmKqtw== 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b] 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQA+oyxm9+xNDRAAhFXmvJv+YKWpQa4MG10lPA== 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.4] 2024-04-27T07:03:55.732 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBOoyxm+BeLKhAAgVHFVXecbZN0h4L6JR6i2A== 2024-04-27T07:03:55.733 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.5] 2024-04-27T07:03:55.733 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBRoyxmCoyNMBAATx0/vNa76EqzZVzqe7aEVg== 2024-04-27T07:03:55.733 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.6] 2024-04-27T07:03:55.733 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBUoyxmMtpIOBAACHVu23d6W1P51eipJYpUzg== 2024-04-27T07:03:55.733 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.7] 2024-04-27T07:03:55.733 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBYoyxm5EfTAxAA004KGJLbiMGRzrS9Q9PofQ== 2024-04-27T07:03:55.733 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-04-27T07:03:55.733 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQA+oyxmgrAhExAAceKsj8G+Q/XXODrJbKPZ0Q== 2024-04-27T07:03:55.734 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.y] 2024-04-27T07:03:55.734 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA+oyxm+NugABAAzPlk/x6yFXaTG5J0uLdcOA== 2024-04-27T07:03:55.735 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.a] 2024-04-27T07:03:55.735 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA+oyxmeosVBxAA64Na7IWgq/9JX8IryFKwqQ== 2024-04-27T07:03:55.735 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.0] 2024-04-27T07:03:55.735 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBBoyxm9cBBEhAA+7TsG1DI1x9B58lFniRTCA== 2024-04-27T07:03:55.735 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.1] 2024-04-27T07:03:55.735 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBEoyxmUkfXFxAAeOVSfqw19vxB9lh6A+B2mw== 2024-04-27T07:03:55.735 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.2] 2024-04-27T07:03:55.735 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBHoyxm05j6HRAASwIItsLH5QWhSJJua+scaQ== 2024-04-27T07:03:55.735 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.3] 2024-04-27T07:03:55.735 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBKoyxmIUTjJRAAXn6H3GX7kIiyF6rxixzGxg== 2024-04-27T07:03:55.736 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.x] 2024-04-27T07:03:55.736 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA+oyxmryeZAxAAbc4+8Txh5jr/zMCfbmKqtw== 2024-04-27T07:03:55.736 INFO:teuthology.orchestra.run.smithi053.stdout:[mds.b] 2024-04-27T07:03:55.736 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA+oyxm9+xNDRAAhFXmvJv+YKWpQa4MG10lPA== 2024-04-27T07:03:55.736 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.4] 2024-04-27T07:03:55.736 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBOoyxm+BeLKhAAgVHFVXecbZN0h4L6JR6i2A== 2024-04-27T07:03:55.736 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.5] 2024-04-27T07:03:55.736 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBRoyxmCoyNMBAATx0/vNa76EqzZVzqe7aEVg== 2024-04-27T07:03:55.736 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.6] 2024-04-27T07:03:55.736 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBUoyxmMtpIOBAACHVu23d6W1P51eipJYpUzg== 2024-04-27T07:03:55.737 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.7] 2024-04-27T07:03:55.737 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBYoyxm5EfTAxAA004KGJLbiMGRzrS9Q9PofQ== 2024-04-27T07:03:55.737 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-04-27T07:03:55.737 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQA+oyxmgrAhExAAceKsj8G+Q/XXODrJbKPZ0Q== 2024-04-27T07:03:55.738 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T07:03:55.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T07:03:55.872 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:03:55.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:03:55.959 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:55.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.028 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.117 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.163 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.202 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.251 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T07:03:56.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T07:03:56.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:03:56.370 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-27T07:03:56.421 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.546 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.593 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.714 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T07:03:56.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T07:03:56.806 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T07:03:56.855 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-27T07:03:56.856 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-27T07:03:56.867 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-27T07:03:56.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-27T07:03:57.011 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-27T07:03:57.026 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-27T07:03:57.121 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-27T07:03:57.133 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-27T07:03:57.191 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-27T07:03:57.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-27T07:03:57.291 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:57.294 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-27T07:03:57.341 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-27T07:03:57.341 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-27T07:03:57.341 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-27T07:03:57.344 INFO:tasks.ceph.mon.a:Started 2024-04-27T07:03:57.344 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-27T07:03:57.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-27T07:03:57.386 INFO:tasks.ceph.mon.b:Started 2024-04-27T07:03:57.387 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-27T07:03:57.387 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-27T07:03:57.389 INFO:tasks.ceph.mon.c:Started 2024-04-27T07:03:57.389 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-27T07:03:57.389 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-27T07:03:57.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-27T07:03:57.392 INFO:tasks.ceph.mgr.y:Started 2024-04-27T07:03:57.392 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-27T07:03:57.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-27T07:03:57.394 INFO:tasks.ceph.mgr.x:Started 2024-04-27T07:03:57.394 INFO:tasks.ceph:Setting crush tunables to default 2024-04-27T07:03:57.394 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd crush tunables default 2024-04-27T07:03:57.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a7168640 1 Processor -- start 2024-04-27T07:03:57.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a7168640 1 -- start start 2024-04-27T07:03:57.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a7168640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f01a00b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:57.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a7168640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 0x7f01a014eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:57.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a7168640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01a0072f10 con 0x7f01a0144f80 2024-04-27T07:03:57.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a7168640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f01a0073180 con 0x7f01a0147f10 2024-04-27T07:03:57.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a7168640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f01a006e3e0 con 0x7f01a006e000 2024-04-27T07:03:57.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a4f04640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f01a00b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:57.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a5705640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 0x7f01a014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:57.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a4f04640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f01a00b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:52854/0 (socket says 172.21.15.12:52854) 2024-04-27T07:03:57.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a4f04640 1 -- 172.21.15.12:0/3478265379 learned_addr learned my addr 172.21.15.12:0/3478265379 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:03:57.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a4f04640 1 -- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 msgr2=0x7f01a00b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-27T07:03:57.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a4f04640 1 -- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 msgr2=0x7f01a00b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-27T07:03:57.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.513+0000 7f01a5705640 1 -- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 msgr2=0x7f01a014eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-27T07:03:57.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.509+0000 7f01a4f04640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f01a00b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-27T07:03:57.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.513+0000 7f01a5705640 1 -- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 msgr2=0x7f01a014eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-27T07:03:57.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.513+0000 7f01a5705640 1 --2- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 0x7f01a014eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-27T07:03:57.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.513+0000 7f01a4f04640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f01a00b5f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T07:03:57.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.513+0000 7f01a5705640 1 --2- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 0x7f01a014eb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-27T07:03:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.717+0000 7f01a4f04640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f01a00b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.717+0000 7f01a5705640 1 --2- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 0x7f01a014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.717+0000 7f01a4f04640 1 -- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 msgr2=0x7f01a00b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-27T07:03:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.717+0000 7f01a4f04640 1 -- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 msgr2=0x7f01a00b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-27T07:03:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.717+0000 7f01a4f04640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f01a00b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-27T07:03:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.717+0000 7f01a4f04640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f01a00b5f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-27T07:03:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.717+0000 7f01a5705640 1 -- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 msgr2=0x7f01a014eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-27T07:03:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.717+0000 7f01a5705640 1 -- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 msgr2=0x7f01a014eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-27T07:03:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.717+0000 7f01a5705640 1 --2- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 0x7f01a014eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-27T07:03:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.717+0000 7f01a5705640 1 --2- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 0x7f01a014eb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-27T07:03:57.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.733+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3891391301 0 0) 0x7f01a0072f10 con 0x7f01a0144f80 2024-04-27T07:03:57.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.733+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0190005740 con 0x7f01a0144f80 2024-04-27T07:03:57.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.733+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 <== mon.1 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2466868745 0 0) 0x7f0190005740 con 0x7f01a0144f80 2024-04-27T07:03:57.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.733+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 msgr2=0x7f01a00b5f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:03:57.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.733+0000 7f019f7fe640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f01a00b5f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:57.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.733+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 msgr2=0x7f01a014eb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:03:57.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.733+0000 7f019f7fe640 1 --2- 172.21.15.12:0/3478265379 >> v2:172.21.15.53:3300/0 conn(0x7f01a0147f10 0x7f01a014eb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:57.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.733+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a00b6440 con 0x7f01a0144f80 2024-04-27T07:03:57.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.733+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 <== mon.1 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f018c001680 con 0x7f01a0144f80 2024-04-27T07:03:57.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 >> v1:172.21.15.53:6789/0 conn(0x7f01a0144f80 legacy=0x7f01a0147800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:57.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019f7fe640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f0190005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:57.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019f7fe640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 0x7f0190006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:57.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019f7fe640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0190006f60 0x7f01900073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:57.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0073180 con 0x7f0190006250 2024-04-27T07:03:57.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01a006e3e0 con 0x7f0190006f60 2024-04-27T07:03:57.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f019000ddf0 con 0x7f01a006e000 2024-04-27T07:03:57.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019ffff640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 0x7f0190006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:57.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f01a4f04640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f0190005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:57.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f01a5705640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0190006f60 0x7f01900073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:57.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019ffff640 1 -- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 msgr2=0x7f0190005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:57.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019ffff640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a006e000 0x7f0190005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:57.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019ffff640 1 -- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0190006f60 msgr2=0x7f01900073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:57.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019ffff640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0190006f60 0x7f01900073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:57.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.737+0000 7f019ffff640 1 -- 172.21.15.12:0/3478265379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0190005740 con 0x7f0190006250 2024-04-27T07:03:57.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f019ffff640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 0x7f0190006a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f018c007f90 tx=0x7f018c01cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd1023ce891947f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0b2070 con 0x7f0190006250 2024-04-27T07:03:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018c0a5820 con 0x7f0190006250 2024-04-27T07:03:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f019f7fe640 1 -- 172.21.15.12:0/3478265379 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c007bc0 con 0x7f0190006250 2024-04-27T07:03:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f01a7168640 1 -- 172.21.15.12:0/3478265379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 msgr2=0x7f0190006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f01a7168640 1 --2- 172.21.15.12:0/3478265379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 0x7f0190006a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f018c007f90 tx=0x7f018c01cb20 comp rx=0 tx=0).stop 2024-04-27T07:03:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f01a5705640 1 -- 172.21.15.12:0/3478265379 reap_dead start 2024-04-27T07:03:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f01a7168640 1 -- 172.21.15.12:0/3478265379 shutdown_connections 2024-04-27T07:03:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f01a7168640 1 -- 172.21.15.12:0/3478265379 >> 172.21.15.12:0/3478265379 conn(0x7f01a00b9e80 msgr2=0x7f01a00ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f01a7168640 1 -- 172.21.15.12:0/3478265379 shutdown_connections 2024-04-27T07:03:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.745+0000 7f01a7168640 1 -- 172.21.15.12:0/3478265379 wait complete. 2024-04-27T07:03:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a7168640 1 Processor -- start 2024-04-27T07:03:57.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a7168640 1 -- start start 2024-04-27T07:03:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a7168640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 0x7f01a014b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a7168640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a014c550 0x7f01a01dca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a7168640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01a01dcf40 0x7f01a01dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a7168640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f01a014f070 con 0x7f0190006250 2024-04-27T07:03:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a7168640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00b56c0 con 0x7f01a01dcf40 2024-04-27T07:03:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a7168640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f01a014f1f0 con 0x7f01a014c550 2024-04-27T07:03:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a4f04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 0x7f01a014b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a4f04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 0x7f01a014b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34972/0 (socket says 172.21.15.12:34972) 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a4f04640 1 -- 172.21.15.12:0/621565966 learned_addr learned my addr 172.21.15.12:0/621565966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f019ffff640 1 --2- 172.21.15.12:0/621565966 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a014c550 0x7f01a01dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.749+0000 7f01a5705640 1 --2- 172.21.15.12:0/621565966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01a01dcf40 0x7f01a01dfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f01a4f04640 1 -- 172.21.15.12:0/621565966 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a014c550 msgr2=0x7f01a01dca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f01a4f04640 1 --2- 172.21.15.12:0/621565966 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a014c550 0x7f01a01dca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f01a4f04640 1 -- 172.21.15.12:0/621565966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01a01dcf40 msgr2=0x7f01a01dfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f01a4f04640 1 --2- 172.21.15.12:0/621565966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01a01dcf40 0x7f01a01dfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f01a4f04640 1 -- 172.21.15.12:0/621565966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a01dd3c0 con 0x7f0190006250 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f01a4f04640 1 --2- 172.21.15.12:0/621565966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 0x7f01a014b350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f019400e8c0 tx=0x7f019400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f019d7fa640 1 -- 172.21.15.12:0/621565966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019401a030 con 0x7f0190006250 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f01a7168640 1 -- 172.21.15.12:0/621565966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a014cd80 con 0x7f0190006250 2024-04-27T07:03:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f019d7fa640 1 -- 172.21.15.12:0/621565966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0194003e20 con 0x7f0190006250 2024-04-27T07:03:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f019d7fa640 1 -- 172.21.15.12:0/621565966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019400f040 con 0x7f0190006250 2024-04-27T07:03:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f01a7168640 1 -- 172.21.15.12:0/621565966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a01414e0 con 0x7f0190006250 2024-04-27T07:03:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f019d7fa640 1 -- 172.21.15.12:0/621565966 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f019401d030 con 0x7f0190006250 2024-04-27T07:03:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f019d7fa640 1 -- 172.21.15.12:0/621565966 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f019401f750 con 0x7f0190006250 2024-04-27T07:03:57.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.753+0000 7f01a7168640 1 -- 172.21.15.12:0/621565966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a0069330 con 0x7f0190006250 2024-04-27T07:03:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.757+0000 7f019d7fa640 1 -- 172.21.15.12:0/621565966 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f01a01dd3c0 con 0x7f0190006250 2024-04-27T07:03:57.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.821+0000 7f01a7168640 1 -- 172.21.15.12:0/621565966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f01a01dc620 con 0x7f0190006250 2024-04-27T07:03:57.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.825+0000 7f019d7fa640 1 -- 172.21.15.12:0/621565966 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f019401f950 con 0x7f0190006250 2024-04-27T07:03:57.829 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-04-27T07:03:57.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.825+0000 7f01a7168640 1 -- 172.21.15.12:0/621565966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 msgr2=0x7f01a014b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:57.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.825+0000 7f01a7168640 1 --2- 172.21.15.12:0/621565966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 0x7f01a014b350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f019400e8c0 tx=0x7f019400ed80 comp rx=0 tx=0).stop 2024-04-27T07:03:57.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.825+0000 7f01a7168640 1 -- 172.21.15.12:0/621565966 shutdown_connections 2024-04-27T07:03:57.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.825+0000 7f01a7168640 1 --2- 172.21.15.12:0/621565966 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01a01dcf40 0x7f01a01dfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:57.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.825+0000 7f01a7168640 1 --2- 172.21.15.12:0/621565966 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01a014c550 0x7f01a01dca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:57.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.825+0000 7f01a7168640 1 --2- 172.21.15.12:0/621565966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0190006250 0x7f01a014b350 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:57.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.825+0000 7f01a7168640 1 -- 172.21.15.12:0/621565966 >> 172.21.15.12:0/621565966 conn(0x7f01a00b9e80 msgr2=0x7f01a00b93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.825+0000 7f01a7168640 1 -- 172.21.15.12:0/621565966 shutdown_connections 2024-04-27T07:03:57.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:03:57.825+0000 7f01a7168640 1 -- 172.21.15.12:0/621565966 wait complete. 2024-04-27T07:03:57.848 INFO:tasks.ceph:check_enable_crimson: False 2024-04-27T07:03:57.848 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-27T07:03:57.848 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:57.848 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-27T07:03:57.859 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:57.859 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-27T07:03:57.914 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:57.915 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-27T07:03:57.967 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:03:57.967 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-27T07:03:58.010 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:58.007+0000 7fd11fb8f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-27T07:03:58.020 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:58.020 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-27T07:03:58.034 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:58.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-27T07:03:58.089 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:58.089 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-27T07:03:58.145 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-04-27T07:03:58.145 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-27T07:03:58.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 57fa2165-d30f-4ee5-a2c8-ad17c3948dcb 0 2024-04-27T07:03:58.303 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:03:58.297+0000 7f08fc3fc180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-27T07:03:58.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb906c0e640 1 Processor -- start 2024-04-27T07:03:58.372 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb906c0e640 1 -- start start 2024-04-27T07:03:58.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb906c0e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9001490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.373 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb906c0e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb9001495e0 0x7fb900150430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb906c0e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb900146a40 con 0x7fb9000b7420 2024-04-27T07:03:58.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb906c0e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fb900145f60 con 0x7fb9001495e0 2024-04-27T07:03:58.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb906c0e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fb900146240 con 0x7fb9000b9df0 2024-04-27T07:03:58.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb8fffff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9001490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb8fffff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9001490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:52480/0 (socket says 172.21.15.53:52480) 2024-04-27T07:03:58.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb8fffff640 1 -- 172.21.15.53:0/1607826879 learned_addr learned my addr 172.21.15.53:0/1607826879 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:58.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.367+0000 7fb9051ab640 1 --2- 172.21.15.53:0/1607826879 >> v2:172.21.15.12:3300/0 conn(0x7fb9001495e0 0x7fb900150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb8fffff640 1 -- 172.21.15.53:0/1607826879 >> v2:172.21.15.12:3300/0 conn(0x7fb9001495e0 msgr2=0x7fb900150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:58.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb8fffff640 1 --2- 172.21.15.53:0/1607826879 >> v2:172.21.15.12:3300/0 conn(0x7fb9001495e0 0x7fb900150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb8fffff640 1 -- 172.21.15.53:0/1607826879 >> v1:172.21.15.12:6789/0 conn(0x7fb9000b7420 legacy=0x7fb90006e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:58.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb8fffff640 1 -- 172.21.15.53:0/1607826879 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9000b7800 con 0x7fb9000b9df0 2024-04-27T07:03:58.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb8fffff640 1 --2- 172.21.15.53:0/1607826879 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9001490a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009870 tx=0x7fb8e801c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6f3563ec77ba201 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:58.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb8ff7fe640 1 -- 172.21.15.53:0/1607826879 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80b0070 con 0x7fb9000b9df0 2024-04-27T07:03:58.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb8ff7fe640 1 -- 172.21.15.53:0/1607826879 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e80a4e50 con 0x7fb9000b9df0 2024-04-27T07:03:58.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb8ff7fe640 1 -- 172.21.15.53:0/1607826879 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80ab820 con 0x7fb9000b9df0 2024-04-27T07:03:58.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb906c0e640 1 -- 172.21.15.53:0/1607826879 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 msgr2=0x7fb9001490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb906c0e640 1 --2- 172.21.15.53:0/1607826879 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9001490a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009870 tx=0x7fb8e801c400 comp rx=0 tx=0).stop 2024-04-27T07:03:58.376 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb906c0e640 1 -- 172.21.15.53:0/1607826879 shutdown_connections 2024-04-27T07:03:58.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb906c0e640 1 --2- 172.21.15.53:0/1607826879 >> v2:172.21.15.12:3300/0 conn(0x7fb9001495e0 0x7fb900150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb906c0e640 1 --2- 172.21.15.53:0/1607826879 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9001490a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb906c0e640 1 -- 172.21.15.53:0/1607826879 >> 172.21.15.53:0/1607826879 conn(0x7fb9000ba6d0 msgr2=0x7fb9000baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:58.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb906c0e640 1 -- 172.21.15.53:0/1607826879 shutdown_connections 2024-04-27T07:03:58.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.371+0000 7fb906c0e640 1 -- 172.21.15.53:0/1607826879 wait complete. 2024-04-27T07:03:58.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 Processor -- start 2024-04-27T07:03:58.377 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 -- start start 2024-04-27T07:03:58.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000b7420 0x7fb90009cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9000b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9001495e0 0x7fb90013f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001533a0 con 0x7fb9000b7420 2024-04-27T07:03:58.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb900145d20 con 0x7fb9001495e0 2024-04-27T07:03:58.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fb900153520 con 0x7fb9000b9df0 2024-04-27T07:03:58.380 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fffff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9000b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fffff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9000b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:52490/0 (socket says 172.21.15.53:52490) 2024-04-27T07:03:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fffff640 1 -- 172.21.15.53:0/922170412 learned_addr learned my addr 172.21.15.53:0/922170412 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb9051ab640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9001495e0 0x7fb90013f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb9049aa640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000b7420 0x7fb90009cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fffff640 1 -- 172.21.15.53:0/922170412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9001495e0 msgr2=0x7fb90013f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fffff640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9001495e0 0x7fb90013f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fffff640 1 -- 172.21.15.53:0/922170412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000b7420 msgr2=0x7fb90009cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fffff640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000b7420 0x7fb90009cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.381 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fffff640 1 -- 172.21.15.53:0/922170412 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9000b0550 con 0x7fb9000b9df0 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb9051ab640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9001495e0 0x7fb90013f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb9049aa640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000b7420 0x7fb90009cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fffff640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9000b0010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80ae040 tx=0x7fb8e80191a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fd7fa640 1 -- 172.21.15.53:0/922170412 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80b0040 con 0x7fb9000b9df0 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fd7fa640 1 -- 172.21.15.53:0/922170412 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e80aa680 con 0x7fb9000b9df0 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 -- 172.21.15.53:0/922170412 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90014fa00 con 0x7fb9000b9df0 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fd7fa640 1 -- 172.21.15.53:0/922170412 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e80aaa70 con 0x7fb9000b9df0 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 -- 172.21.15.53:0/922170412 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9000b0ab0 con 0x7fb9000b9df0 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fd7fa640 1 -- 172.21.15.53:0/922170412 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb8e80aac10 con 0x7fb9000b9df0 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb8fd7fa640 1 -- 172.21.15.53:0/922170412 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fb8e80bd740 con 0x7fb9000b9df0 2024-04-27T07:03:58.382 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.375+0000 7fb906c0e640 1 -- 172.21.15.53:0/922170412 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb900069330 con 0x7fb9000b9df0 2024-04-27T07:03:58.390 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.387+0000 7fb8fd7fa640 1 -- 172.21.15.53:0/922170412 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb9000b0550 con 0x7fb9000b9df0 2024-04-27T07:03:58.459 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.455+0000 7fb906c0e640 1 -- 172.21.15.53:0/922170412 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "57fa2165-d30f-4ee5-a2c8-ad17c3948dcb", "id": 0} v 0) -- 0x7fb900144310 con 0x7fb9000b9df0 2024-04-27T07:03:58.466 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.463+0000 7fb8fd7fa640 1 -- 172.21.15.53:0/922170412 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "57fa2165-d30f-4ee5-a2c8-ad17c3948dcb", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fb8e8016110 con 0x7fb9000b9df0 2024-04-27T07:03:58.466 INFO:teuthology.orchestra.run.smithi053.stdout:0 2024-04-27T07:03:58.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.463+0000 7fb906c0e640 1 -- 172.21.15.53:0/922170412 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 msgr2=0x7fb9000b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.463+0000 7fb906c0e640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9000b0010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80ae040 tx=0x7fb8e80191a0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.463+0000 7fb906c0e640 1 -- 172.21.15.53:0/922170412 shutdown_connections 2024-04-27T07:03:58.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.463+0000 7fb906c0e640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9001495e0 0x7fb90013f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.463+0000 7fb906c0e640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb9000b9df0 0x7fb9000b0010 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.467 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.463+0000 7fb906c0e640 1 --2- 172.21.15.53:0/922170412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9000b7420 0x7fb90009cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.463+0000 7fb906c0e640 1 -- 172.21.15.53:0/922170412 >> 172.21.15.53:0/922170412 conn(0x7fb9000ba6d0 msgr2=0x7fb900072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:58.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.463+0000 7fb906c0e640 1 -- 172.21.15.53:0/922170412 shutdown_connections 2024-04-27T07:03:58.468 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.463+0000 7fb906c0e640 1 -- 172.21.15.53:0/922170412 wait complete. 2024-04-27T07:03:58.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 7bdf73e9-6350-46e3-bbfb-45f0ca6776d9 1 2024-04-27T07:03:58.534 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:58.531+0000 7fd11fb8f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-27T07:03:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.603+0000 7f9a0f63a640 1 Processor -- start 2024-04-27T07:03:58.608 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.603+0000 7f9a0f63a640 1 -- start start 2024-04-27T07:03:58.609 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a080b7e30 0x7f9a080ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9a080b7320 0x7f9a080b7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a080b9360 con 0x7f9a080b9d50 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f9a080b7c70 con 0x7f9a080b7320 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f9a0814bf30 con 0x7f9a080b7e30 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0cbd5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a080b7e30 0x7f9a080ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0d3d6640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f9a080b9d50 0x7f9a08147f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36100/0 (socket says 172.21.15.53:36100) 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0d3d6640 1 -- 172.21.15.53:0/3127590276 learned_addr learned my addr 172.21.15.53:0/3127590276 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0cbd5640 1 -- 172.21.15.53:0/3127590276 >> v2:172.21.15.12:3300/0 conn(0x7f9a080b7320 msgr2=0x7f9a080b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0cbd5640 1 --2- 172.21.15.53:0/3127590276 >> v2:172.21.15.12:3300/0 conn(0x7f9a080b7320 0x7f9a080b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0cbd5640 1 -- 172.21.15.53:0/3127590276 >> v1:172.21.15.53:6789/0 conn(0x7f9a080b9d50 legacy=0x7f9a08147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0cbd5640 1 -- 172.21.15.53:0/3127590276 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0814c3c0 con 0x7f9a080b7e30 2024-04-27T07:03:58.610 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f99fffff640 1 -- 172.21.15.53:0/3127590276 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2437715942 0 0) 0x7f9a080b9360 con 0x7f9a080b9d50 2024-04-27T07:03:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0cbd5640 1 --2- 172.21.15.53:0/3127590276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 0x7f9a080ba130 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9a08068270 tx=0x7f99f801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c449b73676133316 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f99fffff640 1 -- 172.21.15.53:0/3127590276 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f80b0070 con 0x7f9a080b7e30 2024-04-27T07:03:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f99fffff640 1 -- 172.21.15.53:0/3127590276 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f80ab600 con 0x7f9a080b7e30 2024-04-27T07:03:58.612 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f99fffff640 1 -- 172.21.15.53:0/3127590276 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f80ab9d0 con 0x7f9a080b7e30 2024-04-27T07:03:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 -- 172.21.15.53:0/3127590276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 msgr2=0x7f9a080ba130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 --2- 172.21.15.53:0/3127590276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 0x7f9a080ba130 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9a08068270 tx=0x7f99f801c3f0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 -- 172.21.15.53:0/3127590276 shutdown_connections 2024-04-27T07:03:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 --2- 172.21.15.53:0/3127590276 >> v2:172.21.15.12:3300/0 conn(0x7f9a080b7320 0x7f9a080b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 --2- 172.21.15.53:0/3127590276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 0x7f9a080ba130 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 -- 172.21.15.53:0/3127590276 >> 172.21.15.53:0/3127590276 conn(0x7f9a080ba6d0 msgr2=0x7f9a080baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 -- 172.21.15.53:0/3127590276 shutdown_connections 2024-04-27T07:03:58.613 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.607+0000 7f9a0f63a640 1 -- 172.21.15.53:0/3127590276 wait complete. 2024-04-27T07:03:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 Processor -- start 2024-04-27T07:03:58.614 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 -- start start 2024-04-27T07:03:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a080b7320 0x7f9a080b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 0x7f9a08141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080b9d50 0x7f9a08144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0814b680 con 0x7f9a080b9d50 2024-04-27T07:03:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0814b500 con 0x7f9a080b7e30 2024-04-27T07:03:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f9a08155050 con 0x7f9a080b7320 2024-04-27T07:03:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0cbd5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 0x7f9a08141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.616 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0d3d6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a080b7320 0x7f9a080b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0d3d6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a080b7320 0x7f9a080b6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:52492/0 (socket says 172.21.15.53:52492) 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0cbd5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 0x7f9a08141f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:51992/0 (socket says 172.21.15.53:51992) 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0d3d6640 1 -- 172.21.15.53:0/1363608481 learned_addr learned my addr 172.21.15.53:0/1363608481 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0dbd7640 1 --2- 172.21.15.53:0/1363608481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080b9d50 0x7f9a08144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0d3d6640 1 -- 172.21.15.53:0/1363608481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 msgr2=0x7f9a08141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0d3d6640 1 --2- 172.21.15.53:0/1363608481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 0x7f9a08141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0d3d6640 1 -- 172.21.15.53:0/1363608481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080b9d50 msgr2=0x7f9a08144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0d3d6640 1 --2- 172.21.15.53:0/1363608481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080b9d50 0x7f9a08144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0d3d6640 1 -- 172.21.15.53:0/1363608481 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a080b4f80 con 0x7f9a080b7320 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0cbd5640 1 --2- 172.21.15.53:0/1363608481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 0x7f9a08141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0d3d6640 1 --2- 172.21.15.53:0/1363608481 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a080b7320 0x7f9a080b6ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c890 tx=0x7f9a0000cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f99fdffb640 1 -- 172.21.15.53:0/1363608481 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00018070 con 0x7f9a080b7320 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f99fdffb640 1 -- 172.21.15.53:0/1363608481 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0000a640 con 0x7f9a080b7320 2024-04-27T07:03:58.617 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 -- 172.21.15.53:0/1363608481 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a080b6360 con 0x7f9a080b7320 2024-04-27T07:03:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f99fdffb640 1 -- 172.21.15.53:0/1363608481 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000aa30 con 0x7f9a080b7320 2024-04-27T07:03:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 -- 172.21.15.53:0/1363608481 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a08145c80 con 0x7f9a080b7320 2024-04-27T07:03:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f9a0f63a640 1 -- 172.21.15.53:0/1363608481 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a08069330 con 0x7f9a080b7320 2024-04-27T07:03:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.611+0000 7f99fdffb640 1 -- 172.21.15.53:0/1363608481 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9a0001b070 con 0x7f9a080b7320 2024-04-27T07:03:58.618 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.615+0000 7f99fdffb640 1 -- 172.21.15.53:0/1363608481 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f9a0001d7f0 con 0x7f9a080b7320 2024-04-27T07:03:58.619 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.615+0000 7f99fdffb640 1 -- 172.21.15.53:0/1363608481 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9a0000abd0 con 0x7f9a080b7320 2024-04-27T07:03:58.690 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.687+0000 7f9a0f63a640 1 -- 172.21.15.53:0/1363608481 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7bdf73e9-6350-46e3-bbfb-45f0ca6776d9", "id": 1} v 0) -- 0x7f9a0806fa20 con 0x7f9a080b7320 2024-04-27T07:03:58.695 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.691+0000 7f99fdffb640 1 -- 172.21.15.53:0/1363608481 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7bdf73e9-6350-46e3-bbfb-45f0ca6776d9", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f9a0000adb0 con 0x7f9a080b7320 2024-04-27T07:03:58.695 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-04-27T07:03:58.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.691+0000 7f9a0f63a640 1 -- 172.21.15.53:0/1363608481 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a080b7320 msgr2=0x7f9a080b6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.691+0000 7f9a0f63a640 1 --2- 172.21.15.53:0/1363608481 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a080b7320 0x7f9a080b6ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c890 tx=0x7f9a0000cd50 comp rx=0 tx=0).stop 2024-04-27T07:03:58.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.691+0000 7f9a0f63a640 1 -- 172.21.15.53:0/1363608481 shutdown_connections 2024-04-27T07:03:58.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.691+0000 7f9a0f63a640 1 --2- 172.21.15.53:0/1363608481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a080b9d50 0x7f9a08144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.691+0000 7f9a0f63a640 1 --2- 172.21.15.53:0/1363608481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a080b7e30 0x7f9a08141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.691+0000 7f9a0f63a640 1 --2- 172.21.15.53:0/1363608481 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a080b7320 0x7f9a080b6ad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.691+0000 7f9a0f63a640 1 -- 172.21.15.53:0/1363608481 >> 172.21.15.53:0/1363608481 conn(0x7f9a080ba6d0 msgr2=0x7f9a08073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:58.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.691+0000 7f9a0f63a640 1 -- 172.21.15.53:0/1363608481 shutdown_connections 2024-04-27T07:03:58.696 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.691+0000 7f9a0f63a640 1 -- 172.21.15.53:0/1363608481 wait complete. 2024-04-27T07:03:58.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 6546def3-1651-4b63-83df-7cd53215ab49 2 2024-04-27T07:03:58.762 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:58.759+0000 7fd11fb8f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-27T07:03:58.802 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:03:58.797+0000 7f08fc3fc180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-27T07:03:58.836 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.831+0000 7f2c58ba9640 1 Processor -- start 2024-04-27T07:03:58.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.831+0000 7f2c58ba9640 1 -- start start 2024-04-27T07:03:58.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.831+0000 7f2c58ba9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2c5414bd00 0x7f2c540b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.831+0000 7f2c58ba9640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c54154b50 0x7f2c54156f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.831+0000 7f2c58ba9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c54068870 con 0x7f2c540b0c60 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.831+0000 7f2c58ba9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f2c54072ad0 con 0x7f2c5414bd00 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.831+0000 7f2c58ba9640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f2c54073400 con 0x7f2c54154b50 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c52d9d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c54154b50 0x7f2c54156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c52d9d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c54154b50 0x7f2c54156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:52498/0 (socket says 172.21.15.53:52498) 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c52d9d640 1 -- 172.21.15.53:0/808743161 learned_addr learned my addr 172.21.15.53:0/808743161 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c5259c640 1 --2- 172.21.15.53:0/808743161 >> v2:172.21.15.12:3300/0 conn(0x7f2c5414bd00 0x7f2c540b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c52d9d640 1 -- 172.21.15.53:0/808743161 >> v2:172.21.15.12:3300/0 conn(0x7f2c5414bd00 msgr2=0x7f2c540b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c52d9d640 1 --2- 172.21.15.53:0/808743161 >> v2:172.21.15.12:3300/0 conn(0x7f2c5414bd00 0x7f2c540b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c52d9d640 1 -- 172.21.15.53:0/808743161 >> v1:172.21.15.12:6789/0 conn(0x7f2c540b0c60 legacy=0x7f2c54067910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.838 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c52d9d640 1 -- 172.21.15.53:0/808743161 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c540736e0 con 0x7f2c54154b50 2024-04-27T07:03:58.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c52d9d640 1 --2- 172.21.15.53:0/808743161 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c54154b50 0x7f2c54156f40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009870 tx=0x7f2c4401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=696fbdf6a883c126 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:58.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c5159a640 1 -- 172.21.15.53:0/808743161 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c440b1070 con 0x7f2c54154b50 2024-04-27T07:03:58.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c5159a640 1 -- 172.21.15.53:0/808743161 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c440a8050 con 0x7f2c54154b50 2024-04-27T07:03:58.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c5159a640 1 -- 172.21.15.53:0/808743161 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c440ac860 con 0x7f2c54154b50 2024-04-27T07:03:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c58ba9640 1 -- 172.21.15.53:0/808743161 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c54154b50 msgr2=0x7f2c54156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c58ba9640 1 --2- 172.21.15.53:0/808743161 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c54154b50 0x7f2c54156f40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009870 tx=0x7f2c4401c4d0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c58ba9640 1 -- 172.21.15.53:0/808743161 shutdown_connections 2024-04-27T07:03:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c58ba9640 1 --2- 172.21.15.53:0/808743161 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c54154b50 0x7f2c54156f40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c58ba9640 1 --2- 172.21.15.53:0/808743161 >> v2:172.21.15.12:3300/0 conn(0x7f2c5414bd00 0x7f2c540b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c58ba9640 1 -- 172.21.15.53:0/808743161 >> 172.21.15.53:0/808743161 conn(0x7f2c540b67b0 msgr2=0x7f2c540b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c58ba9640 1 -- 172.21.15.53:0/808743161 shutdown_connections 2024-04-27T07:03:58.841 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.835+0000 7f2c58ba9640 1 -- 172.21.15.53:0/808743161 wait complete. 2024-04-27T07:03:58.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 Processor -- start 2024-04-27T07:03:58.842 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 -- start start 2024-04-27T07:03:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c540b0c60 0x7f2c5414e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c5414bd00 0x7f2c54150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c54154b50 0x7f2c541514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2c540b7380 con 0x7f2c54154b50 2024-04-27T07:03:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54070f80 con 0x7f2c5414bd00 2024-04-27T07:03:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f2c5414ee00 con 0x7f2c540b0c60 2024-04-27T07:03:58.844 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c5259c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c540b0c60 0x7f2c5414e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c51d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c5414bd00 0x7f2c54150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c5259c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c540b0c60 0x7f2c5414e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:52508/0 (socket says 172.21.15.53:52508) 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c5259c640 1 -- 172.21.15.53:0/4262388493 learned_addr learned my addr 172.21.15.53:0/4262388493 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c52d9d640 1 --2- 172.21.15.53:0/4262388493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c54154b50 0x7f2c541514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c5259c640 1 -- 172.21.15.53:0/4262388493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c5414bd00 msgr2=0x7f2c54150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c5259c640 1 --2- 172.21.15.53:0/4262388493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c5414bd00 0x7f2c54150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c5259c640 1 -- 172.21.15.53:0/4262388493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c54154b50 msgr2=0x7f2c541514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c5259c640 1 --2- 172.21.15.53:0/4262388493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c54154b50 0x7f2c541514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c5259c640 1 -- 172.21.15.53:0/4262388493 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c54151a10 con 0x7f2c540b0c60 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c51d9b640 1 --2- 172.21.15.53:0/4262388493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c5414bd00 0x7f2c54150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c52d9d640 1 --2- 172.21.15.53:0/4262388493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c54154b50 0x7f2c541514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c5259c640 1 --2- 172.21.15.53:0/4262388493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c540b0c60 0x7f2c5414e8c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00e930 tx=0x7f2c4c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c4affd640 1 -- 172.21.15.53:0/4262388493 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c017070 con 0x7f2c540b0c60 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c4affd640 1 -- 172.21.15.53:0/4262388493 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c4c00ca00 con 0x7f2c540b0c60 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 -- 172.21.15.53:0/4262388493 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c541527c0 con 0x7f2c540b0c60 2024-04-27T07:03:58.845 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c4affd640 1 -- 172.21.15.53:0/4262388493 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c00cdb0 con 0x7f2c540b0c60 2024-04-27T07:03:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 -- 172.21.15.53:0/4262388493 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c54150a10 con 0x7f2c540b0c60 2024-04-27T07:03:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c4affd640 1 -- 172.21.15.53:0/4262388493 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c4c01a070 con 0x7f2c540b0c60 2024-04-27T07:03:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c4affd640 1 -- 172.21.15.53:0/4262388493 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f2c4c006660 con 0x7f2c540b0c60 2024-04-27T07:03:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.839+0000 7f2c58ba9640 1 -- 172.21.15.53:0/4262388493 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c54069330 con 0x7f2c540b0c60 2024-04-27T07:03:58.846 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.843+0000 7f2c4affd640 1 -- 172.21.15.53:0/4262388493 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2c4c006020 con 0x7f2c540b0c60 2024-04-27T07:03:58.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.911+0000 7f2c58ba9640 1 -- 172.21.15.53:0/4262388493 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6546def3-1651-4b63-83df-7cd53215ab49", "id": 2} v 0) -- 0x7f2c54070d60 con 0x7f2c540b0c60 2024-04-27T07:03:58.922 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.919+0000 7f2c4affd640 1 -- 172.21.15.53:0/4262388493 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6546def3-1651-4b63-83df-7cd53215ab49", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f2c4c0059a0 con 0x7f2c540b0c60 2024-04-27T07:03:58.922 INFO:teuthology.orchestra.run.smithi053.stdout:2 2024-04-27T07:03:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.919+0000 7f2c58ba9640 1 -- 172.21.15.53:0/4262388493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c540b0c60 msgr2=0x7f2c5414e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.919+0000 7f2c58ba9640 1 --2- 172.21.15.53:0/4262388493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c540b0c60 0x7f2c5414e8c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00e930 tx=0x7f2c4c00edf0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.919+0000 7f2c58ba9640 1 -- 172.21.15.53:0/4262388493 shutdown_connections 2024-04-27T07:03:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.919+0000 7f2c58ba9640 1 --2- 172.21.15.53:0/4262388493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c54154b50 0x7f2c541514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.919+0000 7f2c58ba9640 1 --2- 172.21.15.53:0/4262388493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c5414bd00 0x7f2c54150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.923 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.919+0000 7f2c58ba9640 1 --2- 172.21.15.53:0/4262388493 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2c540b0c60 0x7f2c5414e8c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:58.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.919+0000 7f2c58ba9640 1 -- 172.21.15.53:0/4262388493 >> 172.21.15.53:0/4262388493 conn(0x7f2c540b67b0 msgr2=0x7f2c5414a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:58.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.919+0000 7f2c58ba9640 1 -- 172.21.15.53:0/4262388493 shutdown_connections 2024-04-27T07:03:58.924 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:58.919+0000 7f2c58ba9640 1 -- 172.21.15.53:0/4262388493 wait complete. 2024-04-27T07:03:58.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new ec12ea7e-5e96-4e37-b733-d997a7f2cb0f 3 2024-04-27T07:03:59.008 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:03:59.001+0000 7f08fc3fc180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-27T07:03:59.056 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.051+0000 7fe903945640 1 Processor -- start 2024-04-27T07:03:59.057 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.051+0000 7fe903945640 1 -- start start 2024-04-27T07:03:59.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0b7420 0x7fe8fc06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe8fc0ba250 0x7fe8fc150430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8fc146a40 con 0x7fe8fc0b9df0 2024-04-27T07:03:59.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fe8fc145f60 con 0x7fe8fc0ba250 2024-04-27T07:03:59.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe8fc146240 con 0x7fe8fc0b7420 2024-04-27T07:03:59.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe900ee0640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fe8fc0b9df0 0x7fe8fc1490a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36112/0 (socket says 172.21.15.53:36112) 2024-04-27T07:03:59.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe9016e1640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0b7420 0x7fe8fc06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.058 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe900ee0640 1 -- 172.21.15.53:0/1929420659 learned_addr learned my addr 172.21.15.53:0/1929420659 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-04-27T07:03:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe901ee2640 1 --2- 172.21.15.53:0/1929420659 >> v2:172.21.15.12:3300/0 conn(0x7fe8fc0ba250 0x7fe8fc150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe9016e1640 1 -- 172.21.15.53:0/1929420659 >> v2:172.21.15.12:3300/0 conn(0x7fe8fc0ba250 msgr2=0x7fe8fc150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe9016e1640 1 --2- 172.21.15.53:0/1929420659 >> v2:172.21.15.12:3300/0 conn(0x7fe8fc0ba250 0x7fe8fc150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe8f3fff640 1 -- 172.21.15.53:0/1929420659 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2242446462 0 0) 0x7fe8fc146a40 con 0x7fe8fc0b9df0 2024-04-27T07:03:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe9016e1640 1 -- 172.21.15.53:0/1929420659 >> v1:172.21.15.53:6789/0 conn(0x7fe8fc0b9df0 legacy=0x7fe8fc1490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.059 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe9016e1640 1 -- 172.21.15.53:0/1929420659 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc1497b0 con 0x7fe8fc0b7420 2024-04-27T07:03:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe9016e1640 1 --2- 172.21.15.53:0/1929420659 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0b7420 0x7fe8fc06e520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80097c0 tx=0x7fe8f801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61be27ee8f6842f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe8f3fff640 1 -- 172.21.15.53:0/1929420659 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f80b0070 con 0x7fe8fc0b7420 2024-04-27T07:03:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe8f3fff640 1 -- 172.21.15.53:0/1929420659 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f80ab600 con 0x7fe8fc0b7420 2024-04-27T07:03:59.060 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe8f3fff640 1 -- 172.21.15.53:0/1929420659 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f80ab9d0 con 0x7fe8fc0b7420 2024-04-27T07:03:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 -- 172.21.15.53:0/1929420659 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0b7420 msgr2=0x7fe8fc06e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 --2- 172.21.15.53:0/1929420659 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0b7420 0x7fe8fc06e520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80097c0 tx=0x7fe8f801c380 comp rx=0 tx=0).stop 2024-04-27T07:03:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 -- 172.21.15.53:0/1929420659 shutdown_connections 2024-04-27T07:03:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 --2- 172.21.15.53:0/1929420659 >> v2:172.21.15.12:3300/0 conn(0x7fe8fc0ba250 0x7fe8fc150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.061 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 --2- 172.21.15.53:0/1929420659 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0b7420 0x7fe8fc06e520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 -- 172.21.15.53:0/1929420659 >> 172.21.15.53:0/1929420659 conn(0x7fe8fc0ba6d0 msgr2=0x7fe8fc0baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 -- 172.21.15.53:0/1929420659 shutdown_connections 2024-04-27T07:03:59.062 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.055+0000 7fe903945640 1 -- 172.21.15.53:0/1929420659 wait complete. 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 Processor -- start 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 -- start start 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8fc0b7420 0x7fe8fc0b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8fc0b9df0 0x7fe8fc1e1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0ba250 0x7fe8fc13ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc153190 con 0x7fe8fc0b9df0 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc145d20 con 0x7fe8fc0b7420 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe8fc153310 con 0x7fe8fc0ba250 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe9016e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8fc0b7420 0x7fe8fc0b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe901ee2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0ba250 0x7fe8fc13ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe9016e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8fc0b7420 0x7fe8fc0b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52022/0 (socket says 172.21.15.53:52022) 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe9016e1640 1 -- 172.21.15.53:0/2919221642 learned_addr learned my addr 172.21.15.53:0/2919221642 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:59.065 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe900ee0640 1 --2- 172.21.15.53:0/2919221642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8fc0b9df0 0x7fe8fc1e1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe901ee2640 1 -- 172.21.15.53:0/2919221642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8fc0b7420 msgr2=0x7fe8fc0b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe901ee2640 1 --2- 172.21.15.53:0/2919221642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8fc0b7420 0x7fe8fc0b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe901ee2640 1 -- 172.21.15.53:0/2919221642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8fc0b9df0 msgr2=0x7fe8fc1e1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe901ee2640 1 --2- 172.21.15.53:0/2919221642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8fc0b9df0 0x7fe8fc1e1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe901ee2640 1 -- 172.21.15.53:0/2919221642 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc09da60 con 0x7fe8fc0ba250 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe9016e1640 1 --2- 172.21.15.53:0/2919221642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8fc0b7420 0x7fe8fc0b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe901ee2640 1 --2- 172.21.15.53:0/2919221642 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0ba250 0x7fe8fc13ee00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007d90 tx=0x7fe8f400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe8f1ffb640 1 -- 172.21.15.53:0/2919221642 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f400edd0 con 0x7fe8fc0ba250 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe8f1ffb640 1 -- 172.21.15.53:0/2919221642 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f40039a0 con 0x7fe8fc0ba250 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 -- 172.21.15.53:0/2919221642 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc1e1650 con 0x7fe8fc0ba250 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe8f1ffb640 1 -- 172.21.15.53:0/2919221642 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4003d90 con 0x7fe8fc0ba250 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 -- 172.21.15.53:0/2919221642 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc073dd0 con 0x7fe8fc0ba250 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe903945640 1 -- 172.21.15.53:0/2919221642 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc069330 con 0x7fe8fc0ba250 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe8f1ffb640 1 -- 172.21.15.53:0/2919221642 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8f401b020 con 0x7fe8fc0ba250 2024-04-27T07:03:59.066 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.059+0000 7fe8f1ffb640 1 -- 172.21.15.53:0/2919221642 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe8f401d410 con 0x7fe8fc0ba250 2024-04-27T07:03:59.067 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.063+0000 7fe8f1ffb640 1 -- 172.21.15.53:0/2919221642 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe8f4014420 con 0x7fe8fc0ba250 2024-04-27T07:03:59.134 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.131+0000 7fe903945640 1 -- 172.21.15.53:0/2919221642 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ec12ea7e-5e96-4e37-b733-d997a7f2cb0f", "id": 3} v 0) -- 0x7fe8fc150560 con 0x7fe8fc0ba250 2024-04-27T07:03:59.140 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.135+0000 7fe8f1ffb640 1 -- 172.21.15.53:0/2919221642 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ec12ea7e-5e96-4e37-b733-d997a7f2cb0f", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe8f40045e0 con 0x7fe8fc0ba250 2024-04-27T07:03:59.140 INFO:teuthology.orchestra.run.smithi053.stdout:3 2024-04-27T07:03:59.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.139+0000 7fe903945640 1 -- 172.21.15.53:0/2919221642 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0ba250 msgr2=0x7fe8fc13ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.139+0000 7fe903945640 1 --2- 172.21.15.53:0/2919221642 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0ba250 0x7fe8fc13ee00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007d90 tx=0x7fe8f400e4b0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.139+0000 7fe903945640 1 -- 172.21.15.53:0/2919221642 shutdown_connections 2024-04-27T07:03:59.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.139+0000 7fe903945640 1 --2- 172.21.15.53:0/2919221642 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe8fc0ba250 0x7fe8fc13ee00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.139+0000 7fe903945640 1 --2- 172.21.15.53:0/2919221642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8fc0b9df0 0x7fe8fc1e1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.139+0000 7fe903945640 1 --2- 172.21.15.53:0/2919221642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8fc0b7420 0x7fe8fc0b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.139+0000 7fe903945640 1 -- 172.21.15.53:0/2919221642 >> 172.21.15.53:0/2919221642 conn(0x7fe8fc0ba6d0 msgr2=0x7fe8fc0735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:59.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.139+0000 7fe903945640 1 -- 172.21.15.53:0/2919221642 shutdown_connections 2024-04-27T07:03:59.142 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.139+0000 7fe903945640 1 -- 172.21.15.53:0/2919221642 wait complete. 2024-04-27T07:03:59.157 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 4fb26f06-cac3-4c15-a77e-2826ef9c9781 4 2024-04-27T07:03:59.276 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.271+0000 7f03221f8640 1 Processor -- start 2024-04-27T07:03:59.277 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.271+0000 7f03221f8640 1 -- start start 2024-04-27T07:03:59.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f03221f8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f031c067b70 0x7f031c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f03221f8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b53d0 0x7f031c06bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f03221f8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f031c067f50 con 0x7f031c0b4a10 2024-04-27T07:03:59.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f03221f8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f031c0b5f40 con 0x7f031c067b70 2024-04-27T07:03:59.278 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f03221f8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f031c0b5ab0 con 0x7f031c0b53d0 2024-04-27T07:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f031b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f031c067b70 0x7f031c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f031affd640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b53d0 0x7f031c06bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f031affd640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b53d0 0x7f031c06bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:52522/0 (socket says 172.21.15.53:52522) 2024-04-27T07:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f031affd640 1 -- 172.21.15.53:0/2418481326 learned_addr learned my addr 172.21.15.53:0/2418481326 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f031affd640 1 -- 172.21.15.53:0/2418481326 >> v2:172.21.15.53:3300/0 conn(0x7f031c067b70 msgr2=0x7f031c0b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f031affd640 1 --2- 172.21.15.53:0/2418481326 >> v2:172.21.15.53:3300/0 conn(0x7f031c067b70 0x7f031c0b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f031affd640 1 -- 172.21.15.53:0/2418481326 >> v1:172.21.15.12:6789/0 conn(0x7f031c0b4a10 legacy=0x7f031c06c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f031affd640 1 -- 172.21.15.53:0/2418481326 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c072e70 con 0x7f031c0b53d0 2024-04-27T07:03:59.279 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f031b7fe640 1 --2- 172.21.15.53:0/2418481326 >> v2:172.21.15.53:3300/0 conn(0x7f031c067b70 0x7f031c0b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:03:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.275+0000 7f031affd640 1 --2- 172.21.15.53:0/2418481326 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b53d0 0x7f031c06bb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0304009870 tx=0x7f030401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6358d63c2d504f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031a7fc640 1 -- 172.21.15.53:0/2418481326 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040b0070 con 0x7f031c0b53d0 2024-04-27T07:03:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031a7fc640 1 -- 172.21.15.53:0/2418481326 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03040a4e50 con 0x7f031c0b53d0 2024-04-27T07:03:59.281 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031a7fc640 1 -- 172.21.15.53:0/2418481326 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040ab820 con 0x7f031c0b53d0 2024-04-27T07:03:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- 172.21.15.53:0/2418481326 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b53d0 msgr2=0x7f031c06bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 --2- 172.21.15.53:0/2418481326 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b53d0 0x7f031c06bb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0304009870 tx=0x7f030401c400 comp rx=0 tx=0).stop 2024-04-27T07:03:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- 172.21.15.53:0/2418481326 shutdown_connections 2024-04-27T07:03:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 --2- 172.21.15.53:0/2418481326 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b53d0 0x7f031c06bb80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.282 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 --2- 172.21.15.53:0/2418481326 >> v2:172.21.15.53:3300/0 conn(0x7f031c067b70 0x7f031c0b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- 172.21.15.53:0/2418481326 >> 172.21.15.53:0/2418481326 conn(0x7f031c0b9ca0 msgr2=0x7f031c0ba0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- 172.21.15.53:0/2418481326 shutdown_connections 2024-04-27T07:03:59.283 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- 172.21.15.53:0/2418481326 wait complete. 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 Processor -- start 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- start start 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f031c067b70 0x7f031c0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b4a10 0x7f031c1e07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f031c0b53d0 0x7f031c1e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f031c155220 con 0x7f031c0b53d0 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f031c06c9e0 con 0x7f031c067b70 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f031c1553a0 con 0x7f031c0b4a10 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031affd640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b4a10 0x7f031c1e07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031affd640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b4a10 0x7f031c1e07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:52534/0 (socket says 172.21.15.53:52534) 2024-04-27T07:03:59.286 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031affd640 1 -- 172.21.15.53:0/513326143 learned_addr learned my addr 172.21.15.53:0/513326143 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031bfff640 1 --2- 172.21.15.53:0/513326143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f031c0b53d0 0x7f031c1e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031b7fe640 1 --2- 172.21.15.53:0/513326143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f031c067b70 0x7f031c0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031affd640 1 -- 172.21.15.53:0/513326143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f031c067b70 msgr2=0x7f031c0b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031affd640 1 --2- 172.21.15.53:0/513326143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f031c067b70 0x7f031c0b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031affd640 1 -- 172.21.15.53:0/513326143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f031c0b53d0 msgr2=0x7f031c1e0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031affd640 1 --2- 172.21.15.53:0/513326143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f031c0b53d0 0x7f031c1e0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031affd640 1 -- 172.21.15.53:0/513326143 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c145d50 con 0x7f031c0b4a10 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f031affd640 1 --2- 172.21.15.53:0/513326143 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b4a10 0x7f031c1e07e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f030401cca0 tx=0x7f03040191a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f0313fff640 1 -- 172.21.15.53:0/513326143 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040b9070 con 0x7f031c0b4a10 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f0313fff640 1 -- 172.21.15.53:0/513326143 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03040abd70 con 0x7f031c0b4a10 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f0313fff640 1 -- 172.21.15.53:0/513326143 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040b0040 con 0x7f031c0b4a10 2024-04-27T07:03:59.287 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- 172.21.15.53:0/513326143 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031c14be50 con 0x7f031c0b4a10 2024-04-27T07:03:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f03221f8640 1 -- 172.21.15.53:0/513326143 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f031c1e4580 con 0x7f031c0b4a10 2024-04-27T07:03:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f0313fff640 1 -- 172.21.15.53:0/513326143 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f03040bc070 con 0x7f031c0b4a10 2024-04-27T07:03:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.279+0000 7f0313fff640 1 -- 172.21.15.53:0/513326143 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f03040bd8c0 con 0x7f031c0b4a10 2024-04-27T07:03:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.283+0000 7f03221f8640 1 -- 172.21.15.53:0/513326143 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f031c069330 con 0x7f031c0b4a10 2024-04-27T07:03:59.288 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.283+0000 7f0313fff640 1 -- 172.21.15.53:0/513326143 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0304016110 con 0x7f031c0b4a10 2024-04-27T07:03:59.356 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.351+0000 7f03221f8640 1 -- 172.21.15.53:0/513326143 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4fb26f06-cac3-4c15-a77e-2826ef9c9781", "id": 4} v 0) -- 0x7f031c0b9540 con 0x7f031c0b4a10 2024-04-27T07:03:59.362 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.359+0000 7f0313fff640 1 -- 172.21.15.53:0/513326143 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4fb26f06-cac3-4c15-a77e-2826ef9c9781", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f03040a4df0 con 0x7f031c0b4a10 2024-04-27T07:03:59.362 INFO:teuthology.orchestra.run.smithi053.stdout:4 2024-04-27T07:03:59.363 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.359+0000 7f03221f8640 1 -- 172.21.15.53:0/513326143 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b4a10 msgr2=0x7f031c1e07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.359+0000 7f03221f8640 1 --2- 172.21.15.53:0/513326143 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b4a10 0x7f031c1e07e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f030401cca0 tx=0x7f03040191a0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.359+0000 7f03221f8640 1 -- 172.21.15.53:0/513326143 shutdown_connections 2024-04-27T07:03:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.359+0000 7f03221f8640 1 --2- 172.21.15.53:0/513326143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f031c0b53d0 0x7f031c1e0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.359+0000 7f03221f8640 1 --2- 172.21.15.53:0/513326143 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f031c0b4a10 0x7f031c1e07e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.359+0000 7f03221f8640 1 --2- 172.21.15.53:0/513326143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f031c067b70 0x7f031c0b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.359+0000 7f03221f8640 1 -- 172.21.15.53:0/513326143 >> 172.21.15.53:0/513326143 conn(0x7f031c0b9ca0 msgr2=0x7f031c072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.359+0000 7f03221f8640 1 -- 172.21.15.53:0/513326143 shutdown_connections 2024-04-27T07:03:59.364 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.359+0000 7f03221f8640 1 -- 172.21.15.53:0/513326143 wait complete. 2024-04-27T07:03:59.372 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:59.367+0000 7fd11fb8f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-27T07:03:59.378 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 13c23066-aafd-48ba-894a-31b6ba9f76ef 5 2024-04-27T07:03:59.431 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:59.427+0000 7fd11fb8f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-27T07:03:59.487 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:59.483+0000 7fd11fb8f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-27T07:03:59.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.487+0000 7f8638f90640 1 Processor -- start 2024-04-27T07:03:59.492 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.487+0000 7f8638f90640 1 -- start start 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.487+0000 7f8638f90640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8634148a30 0x7f8634151960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.487+0000 7f8638f90640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86340af170 con 0x7f86340b4a30 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f8638f90640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86340b3d90 con 0x7f86340b52c0 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f8638f90640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f86340b5a60 con 0x7f8634148a30 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863259c640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f86340b52c0 0x7f86340b4320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36126/0 (socket says 172.21.15.53:36126) 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863259c640 1 -- 172.21.15.53:0/1485246849 learned_addr learned my addr 172.21.15.53:0/1485246849 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f8632d9d640 1 --2- 172.21.15.53:0/1485246849 >> v2:172.21.15.12:3300/0 conn(0x7f8634148a30 0x7f8634151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 341019386 0 0) 0x7f86340b3d90 con 0x7f86340b52c0 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f861c003530 con 0x7f86340b52c0 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 <== mon.1 v1:172.21.15.53:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2014856904 0 0) 0x7f861c003530 con 0x7f86340b52c0 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 >> v2:172.21.15.12:3300/0 conn(0x7f8634148a30 msgr2=0x7f8634151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 --2- 172.21.15.53:0/1485246849 >> v2:172.21.15.12:3300/0 conn(0x7f8634148a30 0x7f8634151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.495 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 >> v1:172.21.15.12:6789/0 conn(0x7f86340b4a30 legacy=0x7f8634148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 --> v1:172.21.15.53:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86340b3ab0 con 0x7f86340b52c0 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 <== mon.1 v1:172.21.15.53:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f8624001680 con 0x7f86340b52c0 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 >> v1:172.21.15.53:6789/0 conn(0x7f86340b52c0 legacy=0x7f86340b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f861c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f861c004810 0x7f861c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f861c0052f0 0x7f861c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f86340b5a60 con 0x7f861c0052f0 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f861c00c1c0 con 0x7f861c004810 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f861c00c3c0 con 0x7f861c0039c0 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863259c640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f861c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f8631d9b640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f861c004810 0x7f861c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f8632d9d640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f861c0052f0 0x7f861c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863259c640 1 -- 172.21.15.53:0/1485246849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f861c004810 msgr2=0x7f861c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863259c640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f861c004810 0x7f861c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.496 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863259c640 1 -- 172.21.15.53:0/1485246849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f861c0052f0 msgr2=0x7f861c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:59.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863259c640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f861c0052f0 0x7f861c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863259c640 1 -- 172.21.15.53:0/1485246849 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c003530 con 0x7f861c0039c0 2024-04-27T07:03:59.497 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f8632d9d640 1 -- 172.21.15.53:0/1485246849 reap_dead start 2024-04-27T07:03:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863259c640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f861c003de0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8624001060 tx=0x7f862401cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba56e6d6556c1d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624007d20 con 0x7f861c0039c0 2024-04-27T07:03:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86240a9020 con 0x7f861c0039c0 2024-04-27T07:03:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.491+0000 7f863159a640 1 -- 172.21.15.53:0/1485246849 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624005ae0 con 0x7f861c0039c0 2024-04-27T07:03:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- 172.21.15.53:0/1485246849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 msgr2=0x7f861c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f861c003de0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8624001060 tx=0x7f862401cba0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- 172.21.15.53:0/1485246849 shutdown_connections 2024-04-27T07:03:59.498 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 --2- 172.21.15.53:0/1485246849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f861c003de0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- 172.21.15.53:0/1485246849 >> 172.21.15.53:0/1485246849 conn(0x7f86340bb4f0 msgr2=0x7f86340bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:59.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- 172.21.15.53:0/1485246849 shutdown_connections 2024-04-27T07:03:59.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- 172.21.15.53:0/1485246849 wait complete. 2024-04-27T07:03:59.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 Processor -- start 2024-04-27T07:03:59.499 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- start start 2024-04-27T07:03:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f86340b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86340aff50 0x7f8634144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8634145360 0x7f8634143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f86340b5820 con 0x7f86340aff50 2024-04-27T07:03:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f86340b56a0 con 0x7f8634145360 2024-04-27T07:03:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f86340b4320 con 0x7f861c0039c0 2024-04-27T07:03:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f863259c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f86340b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8632d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8634145360 0x7f8634143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f863259c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f86340b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:52550/0 (socket says 172.21.15.53:52550) 2024-04-27T07:03:59.501 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f863259c640 1 -- 172.21.15.53:0/1263935025 learned_addr learned my addr 172.21.15.53:0/1263935025 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8631d9b640 1 --2- 172.21.15.53:0/1263935025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86340aff50 0x7f8634144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f863259c640 1 -- 172.21.15.53:0/1263935025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8634145360 msgr2=0x7f8634143720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f863259c640 1 --2- 172.21.15.53:0/1263935025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8634145360 0x7f8634143720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f863259c640 1 -- 172.21.15.53:0/1263935025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86340aff50 msgr2=0x7f8634144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f863259c640 1 --2- 172.21.15.53:0/1263935025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86340aff50 0x7f8634144e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f863259c640 1 -- 172.21.15.53:0/1263935025 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86341457e0 con 0x7f861c0039c0 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f863259c640 1 --2- 172.21.15.53:0/1263935025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f86340b21c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f862401c9d0 tx=0x7f861c005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8622ffd640 1 -- 172.21.15.53:0/1263935025 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86240049f0 con 0x7f861c0039c0 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8622ffd640 1 -- 172.21.15.53:0/1263935025 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86240b2070 con 0x7f861c0039c0 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- 172.21.15.53:0/1263935025 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86340b06f0 con 0x7f861c0039c0 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8622ffd640 1 -- 172.21.15.53:0/1263935025 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624004db0 con 0x7f861c0039c0 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- 172.21.15.53:0/1263935025 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f86340b0e40 con 0x7f861c0039c0 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8622ffd640 1 -- 172.21.15.53:0/1263935025 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8624007460 con 0x7f861c0039c0 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8622ffd640 1 -- 172.21.15.53:0/1263935025 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f86240b99b0 con 0x7f861c0039c0 2024-04-27T07:03:59.502 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.495+0000 7f8638f90640 1 -- 172.21.15.53:0/1263935025 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8634069330 con 0x7f861c0039c0 2024-04-27T07:03:59.503 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.499+0000 7f8622ffd640 1 -- 172.21.15.53:0/1263935025 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f86341457e0 con 0x7f861c0039c0 2024-04-27T07:03:59.570 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:59.567+0000 7fd11fb8f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-27T07:03:59.571 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.567+0000 7f8638f90640 1 -- 172.21.15.53:0/1263935025 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "13c23066-aafd-48ba-894a-31b6ba9f76ef", "id": 5} v 0) -- 0x7f86340b34f0 con 0x7f861c0039c0 2024-04-27T07:03:59.578 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.575+0000 7f8622ffd640 1 -- 172.21.15.53:0/1263935025 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "13c23066-aafd-48ba-894a-31b6ba9f76ef", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f86240168b0 con 0x7f861c0039c0 2024-04-27T07:03:59.578 INFO:teuthology.orchestra.run.smithi053.stdout:5 2024-04-27T07:03:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.575+0000 7f8638f90640 1 -- 172.21.15.53:0/1263935025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 msgr2=0x7f86340b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.575+0000 7f8638f90640 1 --2- 172.21.15.53:0/1263935025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f86340b21c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f862401c9d0 tx=0x7f861c005840 comp rx=0 tx=0).stop 2024-04-27T07:03:59.579 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.575+0000 7f8638f90640 1 -- 172.21.15.53:0/1263935025 shutdown_connections 2024-04-27T07:03:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.575+0000 7f8638f90640 1 --2- 172.21.15.53:0/1263935025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8634145360 0x7f8634143720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.575+0000 7f8638f90640 1 --2- 172.21.15.53:0/1263935025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86340aff50 0x7f8634144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.575+0000 7f8638f90640 1 --2- 172.21.15.53:0/1263935025 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f861c0039c0 0x7f86340b21c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.575+0000 7f8638f90640 1 -- 172.21.15.53:0/1263935025 >> 172.21.15.53:0/1263935025 conn(0x7f86340bb4f0 msgr2=0x7f86340ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.575+0000 7f8638f90640 1 -- 172.21.15.53:0/1263935025 shutdown_connections 2024-04-27T07:03:59.580 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.575+0000 7f8638f90640 1 -- 172.21.15.53:0/1263935025 wait complete. 2024-04-27T07:03:59.586 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:03:59.581+0000 7f08fc3fc180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-27T07:03:59.595 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 5f6a8c98-d79d-4d26-b42b-d2f0b332b951 6 2024-04-27T07:03:59.627 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:59.623+0000 7fd11fb8f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-27T07:03:59.643 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:03:59.637+0000 7f08fc3fc180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-27T07:03:59.697 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:03:59.693+0000 7f08fc3fc180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-27T07:03:59.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba0e5ae640 1 Processor -- start 2024-04-27T07:03:59.707 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba0e5ae640 1 -- start start 2024-04-27T07:03:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba0e5ae640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fba080b11b0 0x7fba0813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba0e5ae640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba0813ffb0 0x7fba081503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba0e5ae640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba081460c0 con 0x7fba080731b0 2024-04-27T07:03:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba0e5ae640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fba080b0b90 con 0x7fba080b11b0 2024-04-27T07:03:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba0e5ae640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fba081463c0 con 0x7fba0813ffb0 2024-04-27T07:03:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba07fff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fba080731b0 0x7fba08149030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.53:36128/0 (socket says 172.21.15.53:36128) 2024-04-27T07:03:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba07fff640 1 -- 172.21.15.53:0/3587871656 learned_addr learned my addr 172.21.15.53:0/3587871656 (peer_addr_for_me v1:172.21.15.53:0/0) 2024-04-27T07:03:59.709 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba0cb4b640 1 --2- 172.21.15.53:0/3587871656 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba0813ffb0 0x7fba081503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba077fe640 1 --2- 172.21.15.53:0/3587871656 >> v2:172.21.15.12:3300/0 conn(0x7fba080b11b0 0x7fba0813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba06ffd640 1 -- 172.21.15.53:0/3587871656 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3934871201 0 0) 0x7fba081460c0 con 0x7fba080731b0 2024-04-27T07:03:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.703+0000 7fba06ffd640 1 -- 172.21.15.53:0/3587871656 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9f0003530 con 0x7fba080731b0 2024-04-27T07:03:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0cb4b640 1 -- 172.21.15.53:0/3587871656 >> v2:172.21.15.12:3300/0 conn(0x7fba080b11b0 msgr2=0x7fba0813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0cb4b640 1 --2- 172.21.15.53:0/3587871656 >> v2:172.21.15.12:3300/0 conn(0x7fba080b11b0 0x7fba0813fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0cb4b640 1 -- 172.21.15.53:0/3587871656 >> v1:172.21.15.53:6789/0 conn(0x7fba080731b0 legacy=0x7fba08149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.710 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0cb4b640 1 -- 172.21.15.53:0/3587871656 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba080b4800 con 0x7fba0813ffb0 2024-04-27T07:03:59.711 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fd11fb8f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-27T07:03:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0cb4b640 1 --2- 172.21.15.53:0/3587871656 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba0813ffb0 0x7fba081503c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fba00009d90 tx=0x7fba0001c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58de85d38ed093b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:59.711 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba06ffd640 1 -- 172.21.15.53:0/3587871656 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000b2070 con 0x7fba0813ffb0 2024-04-27T07:03:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba06ffd640 1 -- 172.21.15.53:0/3587871656 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba000a5820 con 0x7fba0813ffb0 2024-04-27T07:03:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba06ffd640 1 -- 172.21.15.53:0/3587871656 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000a5bf0 con 0x7fba0813ffb0 2024-04-27T07:03:59.712 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0e5ae640 1 -- 172.21.15.53:0/3587871656 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba0813ffb0 msgr2=0x7fba081503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0e5ae640 1 --2- 172.21.15.53:0/3587871656 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba0813ffb0 0x7fba081503c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fba00009d90 tx=0x7fba0001c850 comp rx=0 tx=0).stop 2024-04-27T07:03:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0e5ae640 1 -- 172.21.15.53:0/3587871656 shutdown_connections 2024-04-27T07:03:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0e5ae640 1 --2- 172.21.15.53:0/3587871656 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba0813ffb0 0x7fba081503c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0e5ae640 1 --2- 172.21.15.53:0/3587871656 >> v2:172.21.15.12:3300/0 conn(0x7fba080b11b0 0x7fba0813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0e5ae640 1 -- 172.21.15.53:0/3587871656 >> 172.21.15.53:0/3587871656 conn(0x7fba080b9e80 msgr2=0x7fba080ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0e5ae640 1 -- 172.21.15.53:0/3587871656 shutdown_connections 2024-04-27T07:03:59.713 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.707+0000 7fba0e5ae640 1 -- 172.21.15.53:0/3587871656 wait complete. 2024-04-27T07:03:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 Processor -- start 2024-04-27T07:03:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 -- start start 2024-04-27T07:03:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba080731b0 0x7fba081388b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba080b11b0 0x7fba080b9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba0813ffb0 0x7fba081e3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.715 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba08153120 con 0x7fba0813ffb0 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fba08146a40 con 0x7fba080b11b0 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fba081532a0 con 0x7fba080731b0 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba07fff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba080731b0 0x7fba081388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba080b11b0 0x7fba080b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba077fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba080b11b0 0x7fba080b9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52070/0 (socket says 172.21.15.53:52070) 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba077fe640 1 -- 172.21.15.53:0/2823186965 learned_addr learned my addr 172.21.15.53:0/2823186965 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba07fff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba080731b0 0x7fba081388b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:52574/0 (socket says 172.21.15.53:52574) 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0cb4b640 1 --2- 172.21.15.53:0/2823186965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba0813ffb0 0x7fba081e3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba07fff640 1 -- 172.21.15.53:0/2823186965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba080b11b0 msgr2=0x7fba080b9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba07fff640 1 --2- 172.21.15.53:0/2823186965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba080b11b0 0x7fba080b9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba07fff640 1 -- 172.21.15.53:0/2823186965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba0813ffb0 msgr2=0x7fba081e3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba07fff640 1 --2- 172.21.15.53:0/2823186965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba0813ffb0 0x7fba081e3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba07fff640 1 -- 172.21.15.53:0/2823186965 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba080b8ab0 con 0x7fba080731b0 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba07fff640 1 --2- 172.21.15.53:0/2823186965 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba080731b0 0x7fba081388b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc007fd0 tx=0x7fb9fc00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba04ff9640 1 -- 172.21.15.53:0/2823186965 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc017070 con 0x7fba080731b0 2024-04-27T07:03:59.716 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba04ff9640 1 -- 172.21.15.53:0/2823186965 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc005cf0 con 0x7fba080731b0 2024-04-27T07:03:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba04ff9640 1 -- 172.21.15.53:0/2823186965 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0072f0 con 0x7fba080731b0 2024-04-27T07:03:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 -- 172.21.15.53:0/2823186965 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0813dc10 con 0x7fba080731b0 2024-04-27T07:03:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 -- 172.21.15.53:0/2823186965 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba0813e1d0 con 0x7fba080731b0 2024-04-27T07:03:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba04ff9640 1 -- 172.21.15.53:0/2823186965 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb9fc01a070 con 0x7fba080731b0 2024-04-27T07:03:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba04ff9640 1 -- 172.21.15.53:0/2823186965 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fb9fc0129f0 con 0x7fba080731b0 2024-04-27T07:03:59.717 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.711+0000 7fba0e5ae640 1 -- 172.21.15.53:0/2823186965 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba08069330 con 0x7fba080731b0 2024-04-27T07:03:59.718 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.715+0000 7fba04ff9640 1 -- 172.21.15.53:0/2823186965 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb9fc021020 con 0x7fba080731b0 2024-04-27T07:03:59.769 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:59.767+0000 7fd11fb8f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-27T07:03:59.778 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:03:59.773+0000 7f08fc3fc180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-27T07:03:59.787 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.783+0000 7fba0e5ae640 1 -- 172.21.15.53:0/2823186965 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5f6a8c98-d79d-4d26-b42b-d2f0b332b951", "id": 6} v 0) -- 0x7fba08145640 con 0x7fba080731b0 2024-04-27T07:03:59.792 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.787+0000 7fba04ff9640 1 -- 172.21.15.53:0/2823186965 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5f6a8c98-d79d-4d26-b42b-d2f0b332b951", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fb9fc0075c0 con 0x7fba080731b0 2024-04-27T07:03:59.793 INFO:teuthology.orchestra.run.smithi053.stdout:6 2024-04-27T07:03:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.791+0000 7fba0e5ae640 1 -- 172.21.15.53:0/2823186965 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba080731b0 msgr2=0x7fba081388b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.791+0000 7fba0e5ae640 1 --2- 172.21.15.53:0/2823186965 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba080731b0 0x7fba081388b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc007fd0 tx=0x7fb9fc00c890 comp rx=0 tx=0).stop 2024-04-27T07:03:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.791+0000 7fba0e5ae640 1 -- 172.21.15.53:0/2823186965 shutdown_connections 2024-04-27T07:03:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.791+0000 7fba0e5ae640 1 --2- 172.21.15.53:0/2823186965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba0813ffb0 0x7fba081e3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.791+0000 7fba0e5ae640 1 --2- 172.21.15.53:0/2823186965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba080b11b0 0x7fba080b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.791+0000 7fba0e5ae640 1 --2- 172.21.15.53:0/2823186965 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba080731b0 0x7fba081388b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.791+0000 7fba0e5ae640 1 -- 172.21.15.53:0/2823186965 >> 172.21.15.53:0/2823186965 conn(0x7fba080b9e80 msgr2=0x7fba0806e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:59.794 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.791+0000 7fba0e5ae640 1 -- 172.21.15.53:0/2823186965 shutdown_connections 2024-04-27T07:03:59.795 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.791+0000 7fba0e5ae640 1 -- 172.21.15.53:0/2823186965 wait complete. 2024-04-27T07:03:59.809 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 4792d49e-3f64-40db-ba32-c1975da10ee5 7 2024-04-27T07:03:59.833 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:03:59.829+0000 7f08fc3fc180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-27T07:03:59.914 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:03:59.909+0000 7f08fc3fc180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-27T07:03:59.930 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:59.927+0000 7fd11fb8f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-27T07:03:59.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01e017a640 1 Processor -- start 2024-04-27T07:03:59.968 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01e017a640 1 -- start start 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01e017a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f01d80af770 0x7f01d8144f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01e017a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f01d80b6a50 0x7f01d8145440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01e017a640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80b0470 0x7f01d809c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01e017a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f01d80b0990 con 0x7f01d80af770 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01e017a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f01d80b0c70 con 0x7f01d80b6a50 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01e017a640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f01d80afb50 con 0x7f01d80b0470 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01dd715640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f01d80b6a50 0x7f01d8145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01de717640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80b0470 0x7f01d809c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01dd715640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f01d80b6a50 0x7f01d8145440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52078/0 (socket says 172.21.15.53:52078) 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01de717640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80b0470 0x7f01d809c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.53:52580/0 (socket says 172.21.15.53:52580) 2024-04-27T07:03:59.970 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.963+0000 7f01dd715640 1 -- 172.21.15.53:0/1355580254 learned_addr learned my addr 172.21.15.53:0/1355580254 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01ddf16640 1 --2- 172.21.15.53:0/1355580254 >> v2:172.21.15.12:3300/0 conn(0x7f01d80af770 0x7f01d8144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01dd715640 1 -- 172.21.15.53:0/1355580254 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80b0470 msgr2=0x7f01d809c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01dd715640 1 --2- 172.21.15.53:0/1355580254 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80b0470 0x7f01d809c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01dd715640 1 -- 172.21.15.53:0/1355580254 >> v2:172.21.15.12:3300/0 conn(0x7f01d80af770 msgr2=0x7f01d8144f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:03:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01dd715640 1 --2- 172.21.15.53:0/1355580254 >> v2:172.21.15.12:3300/0 conn(0x7f01d80af770 0x7f01d8144f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01dd715640 1 -- 172.21.15.53:0/1355580254 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d80afd00 con 0x7f01d80b6a50 2024-04-27T07:03:59.971 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01de717640 1 --2- 172.21.15.53:0/1355580254 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80b0470 0x7f01d809c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:03:59.972 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:03:59.965+0000 7f08fc3fc180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-27T07:03:59.975 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01dd715640 1 --2- 172.21.15.53:0/1355580254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 0x7f01d8145440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01d809b4d0 tx=0x7f01c801ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6281335c96d49b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01dcf14640 1 -- 172.21.15.53:0/1355580254 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c80b4070 con 0x7f01d80b6a50 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01dcf14640 1 -- 172.21.15.53:0/1355580254 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c80a7b90 con 0x7f01d80b6a50 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01dcf14640 1 -- 172.21.15.53:0/1355580254 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c80af440 con 0x7f01d80b6a50 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01e017a640 1 -- 172.21.15.53:0/1355580254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 msgr2=0x7f01d8145440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01e017a640 1 --2- 172.21.15.53:0/1355580254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 0x7f01d8145440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01d809b4d0 tx=0x7f01c801ebf0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01e017a640 1 -- 172.21.15.53:0/1355580254 shutdown_connections 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01e017a640 1 --2- 172.21.15.53:0/1355580254 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80b0470 0x7f01d809c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01e017a640 1 --2- 172.21.15.53:0/1355580254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 0x7f01d8145440 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01e017a640 1 --2- 172.21.15.53:0/1355580254 >> v2:172.21.15.12:3300/0 conn(0x7f01d80af770 0x7f01d8144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01e017a640 1 -- 172.21.15.53:0/1355580254 >> 172.21.15.53:0/1355580254 conn(0x7f01d80b6ff0 msgr2=0x7f01d80b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01e017a640 1 -- 172.21.15.53:0/1355580254 shutdown_connections 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.967+0000 7f01e017a640 1 -- 172.21.15.53:0/1355580254 wait complete. 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 Processor -- start 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 -- start start 2024-04-27T07:03:59.976 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80af770 0x7f01d8142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01d80b0470 0x7f01d81dc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 0x7f01d81df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f01d8148be0 con 0x7f01d80b0470 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01d8148a60 con 0x7f01d80b6a50 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f01d8148d60 con 0x7f01d80af770 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01de717640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 0x7f01d81df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01ddf16640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80af770 0x7f01d8142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01de717640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 0x7f01d81df9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:52094/0 (socket says 172.21.15.53:52094) 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01de717640 1 -- 172.21.15.53:0/2257570204 learned_addr learned my addr 172.21.15.53:0/2257570204 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01dd715640 1 --2- 172.21.15.53:0/2257570204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01d80b0470 0x7f01d81dc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01de717640 1 -- 172.21.15.53:0/2257570204 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80af770 msgr2=0x7f01d8142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01de717640 1 --2- 172.21.15.53:0/2257570204 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80af770 0x7f01d8142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.977 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01de717640 1 -- 172.21.15.53:0/2257570204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01d80b0470 msgr2=0x7f01d81dc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01de717640 1 --2- 172.21.15.53:0/2257570204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01d80b0470 0x7f01d81dc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01de717640 1 -- 172.21.15.53:0/2257570204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d80b1d50 con 0x7f01d80b6a50 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01ddf16640 1 --2- 172.21.15.53:0/2257570204 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80af770 0x7f01d8142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01de717640 1 --2- 172.21.15.53:0/2257570204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 0x7f01d81df9e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f01d000efb0 tx=0x7f01d000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01ce7fc640 1 -- 172.21.15.53:0/2257570204 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0018020 con 0x7f01d80b6a50 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01ce7fc640 1 -- 172.21.15.53:0/2257570204 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d0014480 con 0x7f01d80b6a50 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01ce7fc640 1 -- 172.21.15.53:0/2257570204 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0014850 con 0x7f01d80b6a50 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 -- 172.21.15.53:0/2257570204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d8147240 con 0x7f01d80b6a50 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 -- 172.21.15.53:0/2257570204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d8071510 con 0x7f01d80b6a50 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01ce7fc640 1 -- 172.21.15.53:0/2257570204 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f01d001b020 con 0x7f01d80b6a50 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01ce7fc640 1 -- 172.21.15.53:0/2257570204 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f01d001de00 con 0x7f01d80b6a50 2024-04-27T07:03:59.978 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.971+0000 7f01e017a640 1 -- 172.21.15.53:0/2257570204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d8072ac0 con 0x7f01d80b6a50 2024-04-27T07:03:59.979 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:03:59.975+0000 7f01ce7fc640 1 -- 172.21.15.53:0/2257570204 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f01d8072ac0 con 0x7f01d80b6a50 2024-04-27T07:03:59.997 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:03:59.991+0000 7fd11fb8f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-27T07:04:00.046 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.043+0000 7f01e017a640 1 -- 172.21.15.53:0/2257570204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4792d49e-3f64-40db-ba32-c1975da10ee5", "id": 7} v 0) -- 0x7f01d8069330 con 0x7f01d80b6a50 2024-04-27T07:04:00.050 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.047+0000 7f01ce7fc640 1 -- 172.21.15.53:0/2257570204 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4792d49e-3f64-40db-ba32-c1975da10ee5", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f01d0022030 con 0x7f01d80b6a50 2024-04-27T07:04:00.051 INFO:teuthology.orchestra.run.smithi053.stdout:7 2024-04-27T07:04:00.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.047+0000 7f01e017a640 1 -- 172.21.15.53:0/2257570204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 msgr2=0x7f01d81df9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:00.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.047+0000 7f01e017a640 1 --2- 172.21.15.53:0/2257570204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 0x7f01d81df9e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f01d000efb0 tx=0x7f01d000c490 comp rx=0 tx=0).stop 2024-04-27T07:04:00.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.047+0000 7f01e017a640 1 -- 172.21.15.53:0/2257570204 shutdown_connections 2024-04-27T07:04:00.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.047+0000 7f01e017a640 1 --2- 172.21.15.53:0/2257570204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01d80b6a50 0x7f01d81df9e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.047+0000 7f01e017a640 1 --2- 172.21.15.53:0/2257570204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01d80b0470 0x7f01d81dc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.047+0000 7f01e017a640 1 --2- 172.21.15.53:0/2257570204 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01d80af770 0x7f01d8142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.047+0000 7f01e017a640 1 -- 172.21.15.53:0/2257570204 >> 172.21.15.53:0/2257570204 conn(0x7f01d80b6ff0 msgr2=0x7f01d809b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:00.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.047+0000 7f01e017a640 1 -- 172.21.15.53:0/2257570204 shutdown_connections 2024-04-27T07:04:00.052 INFO:teuthology.orchestra.run.smithi053.stderr:2024-04-27T07:04:00.047+0000 7f01e017a640 1 -- 172.21.15.53:0/2257570204 wait complete. 2024-04-27T07:04:00.067 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-27T07:04:00.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-27T07:04:00.070 INFO:tasks.ceph.osd.0:Started 2024-04-27T07:04:00.070 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-27T07:04:00.070 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-27T07:04:00.073 INFO:tasks.ceph.osd.1:Started 2024-04-27T07:04:00.073 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-27T07:04:00.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-27T07:04:00.076 INFO:tasks.ceph.osd.2:Started 2024-04-27T07:04:00.076 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-27T07:04:00.076 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-27T07:04:00.079 INFO:tasks.ceph.osd.3:Started 2024-04-27T07:04:00.079 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-27T07:04:00.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-27T07:04:00.081 INFO:tasks.ceph.osd.4:Started 2024-04-27T07:04:00.081 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-27T07:04:00.081 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-27T07:04:00.083 INFO:tasks.ceph.osd.5:Started 2024-04-27T07:04:00.084 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-27T07:04:00.084 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-27T07:04:00.085 INFO:tasks.ceph.osd.6:Started 2024-04-27T07:04:00.086 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-27T07:04:00.086 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-27T07:04:00.087 INFO:tasks.ceph.osd.7:Started 2024-04-27T07:04:00.088 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:00.146 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:00.141+0000 7f08fc3fc180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-27T07:04:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf8dcc640 1 Processor -- start 2024-04-27T07:04:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf8dcc640 1 -- start start 2024-04-27T07:04:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf8dcc640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf4070d60 0x7f8cf4147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf8dcc640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf406e4d0 con 0x7f8cf4148200 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf8dcc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf4148690 con 0x7f8cf40b60b0 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf8dcc640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf4071f40 con 0x7f8cf4070d60 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf2d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8cf4148200 0x7f8cf414eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40346/0 (socket says 172.21.15.12:40346) 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf2d9d640 1 -- 172.21.15.12:0/2960667795 learned_addr learned my addr 172.21.15.12:0/2960667795 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf1d9b640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf4070d60 0x7f8cf4147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2304019060 0 0) 0x7f8cf406e4d0 con 0x7f8cf4148200 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cdc004640 con 0x7f8cf4148200 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2648005935 0 0) 0x7f8cf4148690 con 0x7f8cf40b60b0 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cf406e4d0 con 0x7f8cf40b60b0 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3211622636 0 0) 0x7f8cdc004640 con 0x7f8cf4148200 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf4070d60 msgr2=0x7f8cf4147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf4070d60 0x7f8cf4147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 >> v1:172.21.15.53:6789/0 conn(0x7f8cf40b60b0 legacy=0x7f8cf406ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf40711a0 con 0x7f8cf4148200 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f8cec001a00 con 0x7f8cf4148200 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 >> v1:172.21.15.12:6789/0 conn(0x7f8cf4148200 legacy=0x7f8cf414eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cdc004c90 0x7f8cdc0050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cdc005920 0x7f8cdc005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf4070d60 0x7f8cdc0062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4071f40 con 0x7f8cdc005920 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc00cbc0 con 0x7f8cdc004c90 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8cdc00cdc0 con 0x7f8cf4070d60 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf1d9b640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cdc005920 0x7f8cdc005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf2d9d640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf4070d60 0x7f8cdc0062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf259c640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cdc004c90 0x7f8cdc0050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf1d9b640 1 -- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf4070d60 msgr2=0x7f8cdc0062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf1d9b640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf4070d60 0x7f8cdc0062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf1d9b640 1 -- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cdc004c90 msgr2=0x7f8cdc0050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:00.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf1d9b640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cdc004c90 0x7f8cdc0050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf1d9b640 1 -- 172.21.15.12:0/2960667795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc004640 con 0x7f8cdc005920 2024-04-27T07:04:00.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f8cf259c640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cdc004c90 0x7f8cdc0050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:00.217 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:00.209+0000 7f08fc3fc180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-27T07:04:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf1d9b640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cdc005920 0x7f8cdc005da0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009ab0 tx=0x7f8ce401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c49f982a81097f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40b2070 con 0x7f8cdc005920 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce40a5620 con 0x7f8cdc005920 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf159a640 1 -- 172.21.15.12:0/2960667795 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40096b0 con 0x7f8cdc005920 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/2960667795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cdc005920 msgr2=0x7f8cdc005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 --2- 172.21.15.12:0/2960667795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cdc005920 0x7f8cdc005da0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009ab0 tx=0x7f8ce401ca30 comp rx=0 tx=0).stop 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf2d9d640 1 -- 172.21.15.12:0/2960667795 reap_dead start 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/2960667795 shutdown_connections 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/2960667795 >> 172.21.15.12:0/2960667795 conn(0x7f8cf40b9e80 msgr2=0x7f8cf40ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/2960667795 shutdown_connections 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/2960667795 wait complete. 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 Processor -- start 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 -- start start 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cdc005920 0x7f8cf40ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf40aff80 0x7f8cf40a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf40aa550 0x7f8cf40aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4147720 con 0x7f8cf40aff80 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4073160 con 0x7f8cdc005920 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf40717c0 con 0x7f8cf40aa550 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf40aff80 0x7f8cf40a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf40aff80 0x7f8cf40a9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35036/0 (socket says 172.21.15.12:35036) 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf1d9b640 1 -- 172.21.15.12:0/1664709265 learned_addr learned my addr 172.21.15.12:0/1664709265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf259c640 1 --2- 172.21.15.12:0/1664709265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cdc005920 0x7f8cf40ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf2d9d640 1 --2- 172.21.15.12:0/1664709265 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf40aa550 0x7f8cf40aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf1d9b640 1 -- 172.21.15.12:0/1664709265 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf40aa550 msgr2=0x7f8cf40aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf1d9b640 1 --2- 172.21.15.12:0/1664709265 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf40aa550 0x7f8cf40aa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf1d9b640 1 -- 172.21.15.12:0/1664709265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cdc005920 msgr2=0x7f8cf40ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf1d9b640 1 --2- 172.21.15.12:0/1664709265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cdc005920 0x7f8cf40ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf1d9b640 1 -- 172.21.15.12:0/1664709265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf41dc8a0 con 0x7f8cf40aff80 2024-04-27T07:04:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf2d9d640 1 --2- 172.21.15.12:0/1664709265 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf40aa550 0x7f8cf40aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf1d9b640 1 --2- 172.21.15.12:0/1664709265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf40aff80 0x7f8cf40a9b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8ce401c860 tx=0x7f8cdc006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8ce2ffd640 1 -- 172.21.15.12:0/1664709265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40ba030 con 0x7f8cf40aff80 2024-04-27T07:04:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8ce2ffd640 1 -- 172.21.15.12:0/1664709265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce40a5e50 con 0x7f8cf40aff80 2024-04-27T07:04:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8ce2ffd640 1 -- 172.21.15.12:0/1664709265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40b2040 con 0x7f8cf40aff80 2024-04-27T07:04:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.213+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/1664709265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf41dcb80 con 0x7f8cf40aff80 2024-04-27T07:04:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.217+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/1664709265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf4144df0 con 0x7f8cf40aff80 2024-04-27T07:04:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.217+0000 7f8ce2ffd640 1 -- 172.21.15.12:0/1664709265 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8ce4004020 con 0x7f8cf40aff80 2024-04-27T07:04:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.217+0000 7f8ce2ffd640 1 -- 172.21.15.12:0/1664709265 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f8ce4003c10 con 0x7f8cf40aff80 2024-04-27T07:04:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.217+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/1664709265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf4069330 con 0x7f8cf40aff80 2024-04-27T07:04:00.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.233+0000 7f8ce2ffd640 1 -- 172.21.15.12:0/1664709265 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8ce40c4020 con 0x7f8cf40aff80 2024-04-27T07:04:00.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.289+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/1664709265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cf4147500 con 0x7f8cf40aff80 2024-04-27T07:04:00.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.293+0000 7f8ce2ffd640 1 -- 172.21.15.12:0/1664709265 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f8ce4016740 con 0x7f8cf40aff80 2024-04-27T07:04:00.296 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:00.296 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:00.046535+0000","last_up_change":"0.000000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:00.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.293+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/1664709265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf40aff80 msgr2=0x7f8cf40a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:00.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.293+0000 7f8cf8dcc640 1 --2- 172.21.15.12:0/1664709265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf40aff80 0x7f8cf40a9b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8ce401c860 tx=0x7f8cdc006950 comp rx=0 tx=0).stop 2024-04-27T07:04:00.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.293+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/1664709265 shutdown_connections 2024-04-27T07:04:00.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.293+0000 7f8cf8dcc640 1 --2- 172.21.15.12:0/1664709265 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8cf40aa550 0x7f8cf40aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.293+0000 7f8cf8dcc640 1 --2- 172.21.15.12:0/1664709265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cf40aff80 0x7f8cf40a9b30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.293+0000 7f8cf8dcc640 1 --2- 172.21.15.12:0/1664709265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cdc005920 0x7f8cf40ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:00.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.293+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/1664709265 >> 172.21.15.12:0/1664709265 conn(0x7f8cf40b9e80 msgr2=0x7f8cf406dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:00.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.293+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/1664709265 shutdown_connections 2024-04-27T07:04:00.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:00.293+0000 7f8cf8dcc640 1 -- 172.21.15.12:0/1664709265 wait complete. 2024-04-27T07:04:00.312 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-27T07:04:00.312 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-27T07:04:00.644 INFO:tasks.ceph.mgr.x.smithi053.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-27T07:04:00.644 INFO:tasks.ceph.mgr.x.smithi053.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-27T07:04:00.644 INFO:tasks.ceph.mgr.x.smithi053.stderr: from numpy import show_config as show_numpy_config 2024-04-27T07:04:00.675 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-04-27T07:04:00.671+0000 7fe957e507c0 -1 Falling back to public interface 2024-04-27T07:04:00.681 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-04-27T07:04:00.677+0000 7fc839efb7c0 -1 Falling back to public interface 2024-04-27T07:04:00.690 INFO:tasks.ceph.osd.6.smithi053.stderr:2024-04-27T07:04:00.687+0000 7f257d7097c0 -1 Falling back to public interface 2024-04-27T07:04:00.699 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:00.695+0000 7fd11fb8f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-27T07:04:00.702 INFO:tasks.ceph.osd.7.smithi053.stderr:2024-04-27T07:04:00.699+0000 7f6935eed7c0 -1 Falling back to public interface 2024-04-27T07:04:00.709 INFO:tasks.ceph.osd.3.smithi012.stderr:2024-04-27T07:04:00.705+0000 7fcd643fe7c0 -1 Falling back to public interface 2024-04-27T07:04:00.710 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-27T07:04:00.705+0000 7f0f2b9537c0 -1 Falling back to public interface 2024-04-27T07:04:00.710 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-04-27T07:04:00.705+0000 7efd3b23f7c0 -1 Falling back to public interface 2024-04-27T07:04:00.714 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-04-27T07:04:00.711+0000 7f8fb7c8b7c0 -1 Falling back to public interface 2024-04-27T07:04:00.856 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:00.851+0000 7fd11fb8f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-27T07:04:00.868 INFO:tasks.ceph.mgr.y.smithi012.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-27T07:04:00.868 INFO:tasks.ceph.mgr.y.smithi012.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-27T07:04:00.868 INFO:tasks.ceph.mgr.y.smithi012.stderr: from numpy import show_config as show_numpy_config 2024-04-27T07:04:00.936 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:00.933+0000 7f08fc3fc180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-27T07:04:01.011 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:01.007+0000 7fd11fb8f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-27T07:04:01.078 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:01.075+0000 7fd11fb8f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-27T07:04:01.118 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:01.123 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:01.117+0000 7f08fc3fc180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-27T07:04:01.218 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:01.215+0000 7fd11fb8f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-27T07:04:01.236 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.229+0000 7fdc3743c640 1 Processor -- start 2024-04-27T07:04:01.236 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc3743c640 1 -- start start 2024-04-27T07:04:01.237 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc3743c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc300b5690 0x7fdc30147720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:01.237 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc3743c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc300b5a70 con 0x7fdc3006ebe0 2024-04-27T07:04:01.237 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc3743c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc3006e3e0 con 0x7fdc3013e130 2024-04-27T07:04:01.237 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc3743c640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fdc3006e190 con 0x7fdc300b5690 2024-04-27T07:04:01.237 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc349d7640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdc3006ebe0 0x7fdc300b0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40358/0 (socket says 172.21.15.12:40358) 2024-04-27T07:04:01.237 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc349d7640 1 -- 172.21.15.12:0/2245484735 learned_addr learned my addr 172.21.15.12:0/2245484735 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:01.239 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc351d8640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc300b5690 0x7fdc30147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:01.239 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032082814 0 0) 0x7fdc300b5a70 con 0x7fdc3006ebe0 2024-04-27T07:04:01.239 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc1c003530 con 0x7fdc3006ebe0 2024-04-27T07:04:01.239 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1451314581 0 0) 0x7fdc1c003530 con 0x7fdc3006ebe0 2024-04-27T07:04:01.239 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc300b5690 msgr2=0x7fdc30147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:01.239 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc300b5690 0x7fdc30147720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:01.239 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 >> v1:172.21.15.53:6789/0 conn(0x7fdc3013e130 legacy=0x7fdc3014ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc30147c60 con 0x7fdc3006ebe0 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fdc2c001680 con 0x7fdc3006ebe0 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 >> v1:172.21.15.12:6789/0 conn(0x7fdc3006ebe0 legacy=0x7fdc300b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 0x7fdc1c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c004810 0x7fdc1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc300b5690 0x7fdc1c005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3006e190 con 0x7fdc1c0039f0 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c00bb60 con 0x7fdc1c004810 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1c00bd60 con 0x7fdc300b5690 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc351d8640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 0x7fdc1c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc349d7640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c004810 0x7fdc1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc359d9640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc300b5690 0x7fdc1c005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc351d8640 1 -- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc300b5690 msgr2=0x7fdc1c005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc351d8640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc300b5690 0x7fdc1c005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:01.240 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc351d8640 1 -- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c004810 msgr2=0x7fdc1c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:01.241 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc351d8640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c004810 0x7fdc1c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:01.241 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.233+0000 7fdc351d8640 1 -- 172.21.15.12:0/2245484735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c003530 con 0x7fdc1c0039f0 2024-04-27T07:04:01.242 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc351d8640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 0x7fdc1c003e10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009ac0 tx=0x7fdc2001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b5d6bcff7381f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:01.242 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20009690 con 0x7fdc1c0039f0 2024-04-27T07:04:01.242 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc200a6430 con 0x7fdc1c0039f0 2024-04-27T07:04:01.242 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc27fff640 1 -- 172.21.15.12:0/2245484735 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200a6800 con 0x7fdc1c0039f0 2024-04-27T07:04:01.242 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 -- 172.21.15.12:0/2245484735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 msgr2=0x7fdc1c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:01.242 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 --2- 172.21.15.12:0/2245484735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 0x7fdc1c003e10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009ac0 tx=0x7fdc2001ca50 comp rx=0 tx=0).stop 2024-04-27T07:04:01.242 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc359d9640 1 -- 172.21.15.12:0/2245484735 reap_dead start 2024-04-27T07:04:01.242 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 -- 172.21.15.12:0/2245484735 shutdown_connections 2024-04-27T07:04:01.243 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 -- 172.21.15.12:0/2245484735 >> 172.21.15.12:0/2245484735 conn(0x7fdc3006bb70 msgr2=0x7fdc3006bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:01.243 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 -- 172.21.15.12:0/2245484735 shutdown_connections 2024-04-27T07:04:01.243 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 -- 172.21.15.12:0/2245484735 wait complete. 2024-04-27T07:04:01.245 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 Processor -- start 2024-04-27T07:04:01.245 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 -- start start 2024-04-27T07:04:01.245 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 0x7fdc3009f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:01.245 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc301542f0 0x7fdc30145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:01.245 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc30145850 0x7fdc30132b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:01.245 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30150540 con 0x7fdc1c0039f0 2024-04-27T07:04:01.245 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc301503c0 con 0x7fdc301542f0 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc3743c640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fdc301506c0 con 0x7fdc30145850 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc351d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 0x7fdc3009f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc351d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 0x7fdc3009f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35052/0 (socket says 172.21.15.12:35052) 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc351d8640 1 -- 172.21.15.12:0/2351987009 learned_addr learned my addr 172.21.15.12:0/2351987009 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc359d9640 1 --2- 172.21.15.12:0/2351987009 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc30145850 0x7fdc30132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc349d7640 1 --2- 172.21.15.12:0/2351987009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc301542f0 0x7fdc30145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc351d8640 1 -- 172.21.15.12:0/2351987009 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc30145850 msgr2=0x7fdc30132b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc351d8640 1 --2- 172.21.15.12:0/2351987009 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc30145850 0x7fdc30132b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc351d8640 1 -- 172.21.15.12:0/2351987009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc301542f0 msgr2=0x7fdc30145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc351d8640 1 --2- 172.21.15.12:0/2351987009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc301542f0 0x7fdc30145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc351d8640 1 -- 172.21.15.12:0/2351987009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc30145cd0 con 0x7fdc1c0039f0 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.237+0000 7fdc351d8640 1 --2- 172.21.15.12:0/2351987009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 0x7fdc3009f0a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdc2001cf30 tx=0x7fdc1c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:01.246 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.241+0000 7fdc25ffb640 1 -- 172.21.15.12:0/2351987009 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200a6ce0 con 0x7fdc1c0039f0 2024-04-27T07:04:01.247 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.241+0000 7fdc25ffb640 1 -- 172.21.15.12:0/2351987009 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc200ba440 con 0x7fdc1c0039f0 2024-04-27T07:04:01.247 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.241+0000 7fdc3743c640 1 -- 172.21.15.12:0/2351987009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc300b8680 con 0x7fdc1c0039f0 2024-04-27T07:04:01.247 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.241+0000 7fdc25ffb640 1 -- 172.21.15.12:0/2351987009 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200ba810 con 0x7fdc1c0039f0 2024-04-27T07:04:01.247 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.241+0000 7fdc3743c640 1 -- 172.21.15.12:0/2351987009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc300b7ac0 con 0x7fdc1c0039f0 2024-04-27T07:04:01.247 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.241+0000 7fdc25ffb640 1 -- 172.21.15.12:0/2351987009 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdc200a7410 con 0x7fdc1c0039f0 2024-04-27T07:04:01.247 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.241+0000 7fdc25ffb640 1 -- 172.21.15.12:0/2351987009 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fdc200bad40 con 0x7fdc1c0039f0 2024-04-27T07:04:01.247 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.241+0000 7fdc3743c640 1 -- 172.21.15.12:0/2351987009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc3006a150 con 0x7fdc1c0039f0 2024-04-27T07:04:01.248 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.241+0000 7fdc25ffb640 1 -- 172.21.15.12:0/2351987009 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdc30145cd0 con 0x7fdc1c0039f0 2024-04-27T07:04:01.303 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.297+0000 7fdc3743c640 1 -- 172.21.15.12:0/2351987009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc3013ce20 con 0x7fdc1c0039f0 2024-04-27T07:04:01.304 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.301+0000 7fdc25ffb640 1 -- 172.21.15.12:0/2351987009 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fdc200ac020 con 0x7fdc1c0039f0 2024-04-27T07:04:01.304 INFO:teuthology.misc.health.smithi012.stdout: 2024-04-27T07:04:01.304 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":10,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:00.046535+0000","last_up_change":"0.000000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:01.305 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.301+0000 7fdc3743c640 1 -- 172.21.15.12:0/2351987009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 msgr2=0x7fdc3009f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:01.305 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.301+0000 7fdc3743c640 1 --2- 172.21.15.12:0/2351987009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 0x7fdc3009f0a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdc2001cf30 tx=0x7fdc1c005840 comp rx=0 tx=0).stop 2024-04-27T07:04:01.305 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.301+0000 7fdc3743c640 1 -- 172.21.15.12:0/2351987009 shutdown_connections 2024-04-27T07:04:01.305 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.301+0000 7fdc3743c640 1 --2- 172.21.15.12:0/2351987009 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc30145850 0x7fdc30132b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:01.305 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.301+0000 7fdc3743c640 1 --2- 172.21.15.12:0/2351987009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc301542f0 0x7fdc30145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:01.306 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.301+0000 7fdc3743c640 1 --2- 172.21.15.12:0/2351987009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c0039f0 0x7fdc3009f0a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:01.306 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.301+0000 7fdc3743c640 1 -- 172.21.15.12:0/2351987009 >> 172.21.15.12:0/2351987009 conn(0x7fdc3006bb70 msgr2=0x7fdc300b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:01.306 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.301+0000 7fdc3743c640 1 -- 172.21.15.12:0/2351987009 shutdown_connections 2024-04-27T07:04:01.306 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:01.301+0000 7fdc3743c640 1 -- 172.21.15.12:0/2351987009 wait complete. 2024-04-27T07:04:01.308 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:01.305+0000 7f08fc3fc180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-27T07:04:01.319 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-27T07:04:01.378 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:01.373+0000 7f08fc3fc180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-27T07:04:01.525 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:01.521+0000 7f08fc3fc180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-27T07:04:01.858 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:01.855+0000 7fd11fb8f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-27T07:04:01.938 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:01.935+0000 7fd11fb8f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-27T07:04:02.128 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:02.123+0000 7fd11fb8f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-27T07:04:02.181 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:02.177+0000 7f08fc3fc180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-27T07:04:02.205 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:02.203+0000 7fd11fb8f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-27T07:04:02.259 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:02.253+0000 7f08fc3fc180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-27T07:04:02.449 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:02.445+0000 7f08fc3fc180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-27T07:04:02.511 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:02.507+0000 7fd11fb8f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-27T07:04:02.526 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:02.521+0000 7f08fc3fc180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-27T07:04:02.576 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:02.571+0000 7fd11fb8f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-27T07:04:02.835 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:02.829+0000 7f08fc3fc180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-27T07:04:02.899 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-27T07:04:02.893+0000 7f08fc3fc180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-27T07:04:03.620 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:03.615+0000 7fd0c6ed9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-27T07:04:03.621 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-04-27T07:04:03.619+0000 7fd0c6ed9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-27T07:04:04.321 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-04-27T07:04:04.319+0000 7f8fb7c8b7c0 -1 osd.5 0 log_to_monitors true 2024-04-27T07:04:04.459 INFO:tasks.ceph.osd.7.smithi053.stderr:2024-04-27T07:04:04.455+0000 7f6935eed7c0 -1 osd.7 0 log_to_monitors true 2024-04-27T07:04:04.516 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-04-27T07:04:04.513+0000 7fc839efb7c0 -1 osd.1 0 log_to_monitors true 2024-04-27T07:04:04.581 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-04-27T07:04:04.575+0000 7fe957e507c0 -1 osd.4 0 log_to_monitors true 2024-04-27T07:04:04.586 INFO:tasks.ceph.osd.6.smithi053.stderr:2024-04-27T07:04:04.583+0000 7f257d7097c0 -1 osd.6 0 log_to_monitors true 2024-04-27T07:04:04.622 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-27T07:04:04.617+0000 7f0f2b9537c0 -1 osd.0 0 log_to_monitors true 2024-04-27T07:04:04.651 INFO:tasks.ceph.osd.3.smithi012.stderr:2024-04-27T07:04:04.645+0000 7fcd643fe7c0 -1 osd.3 0 log_to_monitors true 2024-04-27T07:04:04.676 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-04-27T07:04:04.673+0000 7efd3b23f7c0 -1 osd.2 0 log_to_monitors true 2024-04-27T07:04:07.347 INFO:tasks.ceph.osd.7.smithi053.stderr:2024-04-27T07:04:07.342+0000 7f6931e96640 -1 osd.7 0 waiting for initial osdmap 2024-04-27T07:04:07.370 INFO:tasks.ceph.osd.7.smithi053.stderr:2024-04-27T07:04:07.366+0000 7f6929cca640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:04:07.394 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-04-27T07:04:07.390+0000 7fe953df9640 -1 osd.4 0 waiting for initial osdmap 2024-04-27T07:04:07.418 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-04-27T07:04:07.414+0000 7fe94bc2d640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:04:07.426 INFO:tasks.ceph.osd.6.smithi053.stderr:2024-04-27T07:04:07.422+0000 7f2579ec4640 -1 osd.6 0 waiting for initial osdmap 2024-04-27T07:04:07.430 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-04-27T07:04:07.426+0000 7f8fb3c34640 -1 osd.5 0 waiting for initial osdmap 2024-04-27T07:04:07.458 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-04-27T07:04:07.454+0000 7f8faba68640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:04:07.458 INFO:tasks.ceph.osd.6.smithi053.stderr:2024-04-27T07:04:07.454+0000 7f25714e6640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:04:07.916 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-04-27T07:04:07.913+0000 7fc835ea4640 -1 osd.1 0 waiting for initial osdmap 2024-04-27T07:04:07.947 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-04-27T07:04:07.941+0000 7fc82dcd8640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:04:07.948 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-27T07:04:07.941+0000 7f0f278fc640 -1 osd.0 0 waiting for initial osdmap 2024-04-27T07:04:07.971 INFO:tasks.ceph.osd.3.smithi012.stderr:2024-04-27T07:04:07.965+0000 7fcd603a7640 -1 osd.3 0 waiting for initial osdmap 2024-04-27T07:04:07.981 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-27T07:04:07.977+0000 7f0f1f730640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:04:07.987 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-04-27T07:04:07.981+0000 7efd371e8640 -1 osd.2 0 waiting for initial osdmap 2024-04-27T07:04:08.012 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-04-27T07:04:08.009+0000 7efd2f01c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:04:08.020 INFO:tasks.ceph.osd.3.smithi012.stderr:2024-04-27T07:04:08.017+0000 7fcd581db640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T07:04:08.125 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:08.251 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.245+0000 7ffb3cd60640 1 Processor -- start 2024-04-27T07:04:08.251 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.245+0000 7ffb3cd60640 1 -- start start 2024-04-27T07:04:08.252 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3cd60640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ffb380af920 0x7ffb380afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.253 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3cd60640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb380b02f0 con 0x7ffb380b2aa0 2024-04-27T07:04:08.253 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3cd60640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb380b2e80 con 0x7ffb381463c0 2024-04-27T07:04:08.253 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3cd60640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ffb380af320 con 0x7ffb380af920 2024-04-27T07:04:08.253 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3659c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ffb380b2aa0 0x7ffb380ac770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34742/0 (socket says 172.21.15.12:34742) 2024-04-27T07:04:08.253 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3659c640 1 -- 172.21.15.12:0/3175488 learned_addr learned my addr 172.21.15.12:0/3175488 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:08.253 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb35d9b640 1 --2- 172.21.15.12:0/3175488 >> v2:172.21.15.53:3300/0 conn(0x7ffb380af920 0x7ffb380afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.253 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3094738720 0 0) 0x7ffb380b02f0 con 0x7ffb380b2aa0 2024-04-27T07:04:08.253 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb20003530 con 0x7ffb380b2aa0 2024-04-27T07:04:08.255 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1220681996 0 0) 0x7ffb20003530 con 0x7ffb380b2aa0 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 >> v2:172.21.15.53:3300/0 conn(0x7ffb380af920 msgr2=0x7ffb380afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 --2- 172.21.15.12:0/3175488 >> v2:172.21.15.53:3300/0 conn(0x7ffb380af920 0x7ffb380afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 >> v1:172.21.15.53:6789/0 conn(0x7ffb381463c0 legacy=0x7ffb381467f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb380adab0 con 0x7ffb380b2aa0 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7ffb28001680 con 0x7ffb380b2aa0 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 >> v1:172.21.15.12:6789/0 conn(0x7ffb380b2aa0 legacy=0x7ffb380ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb200039c0 0x7ffb20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb20004810 0x7ffb20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb200052f0 0x7ffb2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffb380af320 con 0x7ffb20004810 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2000c1c0 con 0x7ffb200052f0 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2000c3c0 con 0x7ffb200039c0 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb35d9b640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb20004810 0x7ffb20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb3659c640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb200039c0 0x7ffb20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.256 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb36d9d640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb200052f0 0x7ffb2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.257 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb35d9b640 1 -- 172.21.15.12:0/3175488 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb200039c0 msgr2=0x7ffb20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:08.257 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb35d9b640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb200039c0 0x7ffb20003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.257 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb35d9b640 1 -- 172.21.15.12:0/3175488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb200052f0 msgr2=0x7ffb2000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:08.257 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb35d9b640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb200052f0 0x7ffb2000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.257 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb35d9b640 1 -- 172.21.15.12:0/3175488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb20003530 con 0x7ffb20004810 2024-04-27T07:04:08.257 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.249+0000 7ffb36d9d640 1 -- 172.21.15.12:0/3175488 reap_dead start 2024-04-27T07:04:08.260 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb35d9b640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb20004810 0x7ffb20004c90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009ab0 tx=0x7ffb2c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6b28c802daef3f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:08.260 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0096b0 con 0x7ffb20004810 2024-04-27T07:04:08.260 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb2c0a6410 con 0x7ffb20004810 2024-04-27T07:04:08.260 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3559a640 1 -- 172.21.15.12:0/3175488 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0a6800 con 0x7ffb20004810 2024-04-27T07:04:08.260 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- 172.21.15.12:0/3175488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb20004810 msgr2=0x7ffb20004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.260 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb20004810 0x7ffb20004c90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009ab0 tx=0x7ffb2c01ca50 comp rx=0 tx=0).stop 2024-04-27T07:04:08.260 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- 172.21.15.12:0/3175488 shutdown_connections 2024-04-27T07:04:08.260 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 --2- 172.21.15.12:0/3175488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb20004810 0x7ffb20004c90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- 172.21.15.12:0/3175488 >> 172.21.15.12:0/3175488 conn(0x7ffb380b5c30 msgr2=0x7ffb380b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- 172.21.15.12:0/3175488 shutdown_connections 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- 172.21.15.12:0/3175488 wait complete. 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 Processor -- start 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- start start 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb20004810 0x7ffb380ab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb381e1ec0 0x7ffb381e2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb380702b0 0x7ffb38070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffb38146b60 con 0x7ffb381e1ec0 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffb381469e0 con 0x7ffb380702b0 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7ffb38146ce0 con 0x7ffb20004810 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb35d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb381e1ec0 0x7ffb381e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb35d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb381e1ec0 0x7ffb381e2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38204/0 (socket says 172.21.15.12:38204) 2024-04-27T07:04:08.261 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb35d9b640 1 -- 172.21.15.12:0/2313512572 learned_addr learned my addr 172.21.15.12:0/2313512572 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3659c640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb20004810 0x7ffb380ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb36d9d640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb380702b0 0x7ffb38070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb35d9b640 1 -- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb20004810 msgr2=0x7ffb380ab200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb35d9b640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb20004810 0x7ffb380ab200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb35d9b640 1 -- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb380702b0 msgr2=0x7ffb38070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb35d9b640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb380702b0 0x7ffb38070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb35d9b640 1 -- 172.21.15.12:0/2313512572 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb381e2880 con 0x7ffb381e1ec0 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb36d9d640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb380702b0 0x7ffb38070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb35d9b640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb381e1ec0 0x7ffb381e2340 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c01c880 tx=0x7ffb2c0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb26ffd640 1 -- 172.21.15.12:0/2313512572 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0be070 con 0x7ffb381e1ec0 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb26ffd640 1 -- 172.21.15.12:0/2313512572 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb2c00f000 con 0x7ffb381e1ec0 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb26ffd640 1 -- 172.21.15.12:0/2313512572 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0b9410 con 0x7ffb381e1ec0 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- 172.21.15.12:0/2313512572 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb381d1ec0 con 0x7ffb381e1ec0 2024-04-27T07:04:08.262 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.253+0000 7ffb3cd60640 1 -- 172.21.15.12:0/2313512572 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb38072940 con 0x7ffb381e1ec0 2024-04-27T07:04:08.263 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.257+0000 7ffb3cd60640 1 -- 172.21.15.12:0/2313512572 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3806a150 con 0x7ffb381e1ec0 2024-04-27T07:04:08.263 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.257+0000 7ffb26ffd640 1 -- 172.21.15.12:0/2313512572 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7ffb2c0b03f0 con 0x7ffb381e1ec0 2024-04-27T07:04:08.264 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.257+0000 7ffb26ffd640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ffb180682f0 0x7ffb1806a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.264 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.257+0000 7ffb26ffd640 1 -- 172.21.15.12:0/2313512572 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7ffb2c11f410 con 0x7ffb381e1ec0 2024-04-27T07:04:08.264 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.257+0000 7ffb3659c640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ffb180682f0 0x7ffb1806a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.264 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.257+0000 7ffb3659c640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ffb180682f0 0x7ffb1806a7b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb381476a0 tx=0x7ffb28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:08.266 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.261+0000 7ffb26ffd640 1 -- 172.21.15.12:0/2313512572 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb381e2880 con 0x7ffb381e1ec0 2024-04-27T07:04:08.439 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.433+0000 7ffb3cd60640 1 -- 172.21.15.12:0/2313512572 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb381d1be0 con 0x7ffb381e1ec0 2024-04-27T07:04:08.441 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb26ffd640 1 -- 172.21.15.12:0/2313512572 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10849 (secure 0 0 0) 0x7ffb2c0ac020 con 0x7ffb381e1ec0 2024-04-27T07:04:08.441 INFO:teuthology.misc.health.smithi012.stdout: 2024-04-27T07:04:08.442 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":13,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:08.349154+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 -- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ffb180682f0 msgr2=0x7ffb1806a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ffb180682f0 0x7ffb1806a7b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb381476a0 tx=0x7ffb28009040 comp rx=0 tx=0).stop 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 -- 172.21.15.12:0/2313512572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb381e1ec0 msgr2=0x7ffb381e2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb381e1ec0 0x7ffb381e2340 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c01c880 tx=0x7ffb2c0197f0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 -- 172.21.15.12:0/2313512572 shutdown_connections 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ffb180682f0 0x7ffb1806a7b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffb380702b0 0x7ffb38070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb381e1ec0 0x7ffb381e2340 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 --2- 172.21.15.12:0/2313512572 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffb20004810 0x7ffb380ab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 -- 172.21.15.12:0/2313512572 >> 172.21.15.12:0/2313512572 conn(0x7ffb380b5c30 msgr2=0x7ffb38148630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 -- 172.21.15.12:0/2313512572 shutdown_connections 2024-04-27T07:04:08.444 INFO:teuthology.misc.health.smithi012.stderr:2024-04-27T07:04:08.437+0000 7ffb3cd60640 1 -- 172.21.15.12:0/2313512572 wait complete. 2024-04-27T07:04:08.459 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-27T07:04:08.459 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-27T07:04:08.459 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-27T07:04:08.460 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-27T07:04:08.462 INFO:tasks.ceph.mds.a:Started 2024-04-27T07:04:08.462 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-27T07:04:08.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-27T07:04:08.465 INFO:tasks.ceph.mds.b:Started 2024-04-27T07:04:08.465 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-27T07:04:08.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:08.530 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2024-04-27T07:04:08.557 INFO:tasks.ceph.mds.b.smithi053.stdout:starting mds.b at 2024-04-27T07:04:08.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f1de09640 1 Processor -- start 2024-04-27T07:04:08.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f1de09640 1 -- start start 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f1de09640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f180b5ae0 0x7f7f180b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f1de09640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f180b7560 con 0x7f7f180b64c0 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f1de09640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f180b68a0 con 0x7f7f18148810 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f1de09640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7f18148ca0 con 0x7f7f180b5ae0 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f177fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7f180b64c0 0x7f7f18141ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34744/0 (socket says 172.21.15.12:34744) 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f177fe640 1 -- 172.21.15.12:0/4190191018 learned_addr learned my addr 172.21.15.12:0/4190191018 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606090614 0 0) 0x7f7f180b7560 con 0x7f7f180b64c0 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f00003530 con 0x7f7f180b64c0 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f16ffd640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f180b5ae0 0x7f7f180b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 154085123 0 0) 0x7f7f00003530 con 0x7f7f180b64c0 2024-04-27T07:04:08.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f180b5ae0 msgr2=0x7f7f180b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f180b5ae0 0x7f7f180b7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 >> v1:172.21.15.53:6789/0 conn(0x7f7f18148810 legacy=0x7f7f18152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f18148010 con 0x7f7f180b64c0 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f7f08001680 con 0x7f7f180b64c0 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 >> v1:172.21.15.12:6789/0 conn(0x7f7f180b64c0 legacy=0x7f7f18141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f000039f0 0x7f7f00003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f180b5ae0 0x7f7f00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f00004da0 0x7f7f00005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0000bc20 con 0x7f7f00004da0 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0000bdd0 con 0x7f7f000039f0 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0000bfd0 con 0x7f7f180b5ae0 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f17fff640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f00004da0 0x7f7f00005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f177fe640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f000039f0 0x7f7f00003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f16ffd640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f180b5ae0 0x7f7f00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.569+0000 7f7f17fff640 1 -- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f180b5ae0 msgr2=0x7f7f00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:08.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f17fff640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f180b5ae0 0x7f7f00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f17fff640 1 -- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f000039f0 msgr2=0x7f7f00003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:08.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f17fff640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f000039f0 0x7f7f00003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f17fff640 1 -- 172.21.15.12:0/4190191018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f00003530 con 0x7f7f00004da0 2024-04-27T07:04:08.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f16ffd640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f180b5ae0 0x7f7f00004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f17fff640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f00004da0 0x7f7f00005220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f10001e40 tx=0x7f7f1001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d0be9c47e29177 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f100b0070 con 0x7f7f00004da0 2024-04-27T07:04:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f10003b80 con 0x7f7f00004da0 2024-04-27T07:04:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f167fc640 1 -- 172.21.15.12:0/4190191018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10001a90 con 0x7f7f00004da0 2024-04-27T07:04:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f1de09640 1 -- 172.21.15.12:0/4190191018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f00004da0 msgr2=0x7f7f00005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f1de09640 1 --2- 172.21.15.12:0/4190191018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f00004da0 0x7f7f00005220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f10001e40 tx=0x7f7f1001c740 comp rx=0 tx=0).stop 2024-04-27T07:04:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f17fff640 1 -- 172.21.15.12:0/4190191018 reap_dead start 2024-04-27T07:04:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f1de09640 1 -- 172.21.15.12:0/4190191018 shutdown_connections 2024-04-27T07:04:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f1de09640 1 -- 172.21.15.12:0/4190191018 >> 172.21.15.12:0/4190191018 conn(0x7f7f18073c00 msgr2=0x7f7f180bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:08.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f1de09640 1 -- 172.21.15.12:0/4190191018 shutdown_connections 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f1de09640 1 -- 172.21.15.12:0/4190191018 wait complete. 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f1de09640 1 Processor -- start 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.573+0000 7f7f1de09640 1 -- start start 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f1de09640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f00004da0 0x7f7f18152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f1de09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f181534c0 0x7f7f18142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f1de09640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f18143380 0x7f7f181441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f1de09640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f181421e0 con 0x7f7f181534c0 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f1de09640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18142360 con 0x7f7f00004da0 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f1de09640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7f18154570 con 0x7f7f18143380 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f16ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f181534c0 0x7f7f18142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f16ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f181534c0 0x7f7f18142e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38238/0 (socket says 172.21.15.12:38238) 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f16ffd640 1 -- 172.21.15.12:0/1939104138 learned_addr learned my addr 172.21.15.12:0/1939104138 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f177fe640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f00004da0 0x7f7f18152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f16ffd640 1 -- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f18143380 msgr2=0x7f7f181441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f16ffd640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f18143380 0x7f7f181441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f16ffd640 1 -- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f00004da0 msgr2=0x7f7f18152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f16ffd640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f00004da0 0x7f7f18152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f16ffd640 1 -- 172.21.15.12:0/1939104138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f18143800 con 0x7f7f181534c0 2024-04-27T07:04:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f177fe640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f00004da0 0x7f7f18152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f16ffd640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f181534c0 0x7f7f18142e40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00f4f0 tx=0x7f7f0c00f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7efffff640 1 -- 172.21.15.12:0/1939104138 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c019070 con 0x7f7f181534c0 2024-04-27T07:04:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7efffff640 1 -- 172.21.15.12:0/1939104138 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0c014870 con 0x7f7f181534c0 2024-04-27T07:04:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f1de09640 1 -- 172.21.15.12:0/1939104138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1813e5e0 con 0x7f7f181534c0 2024-04-27T07:04:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7efffff640 1 -- 172.21.15.12:0/1939104138 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c014c60 con 0x7f7f181534c0 2024-04-27T07:04:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f1de09640 1 -- 172.21.15.12:0/1939104138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f181e0e70 con 0x7f7f181534c0 2024-04-27T07:04:08.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f1de09640 1 -- 172.21.15.12:0/1939104138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f18068c20 con 0x7f7f181534c0 2024-04-27T07:04:08.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7efffff640 1 -- 172.21.15.12:0/1939104138 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7f7f0c01c070 con 0x7f7f181534c0 2024-04-27T07:04:08.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7efffff640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7f040682f0 0x7f7f0406a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7efffff640 1 -- 172.21.15.12:0/1939104138 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f7f0c086790 con 0x7f7f181534c0 2024-04-27T07:04:08.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f177fe640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7f040682f0 0x7f7f0406a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.577+0000 7f7f177fe640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7f040682f0 0x7f7f0406a7b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f08000c10 tx=0x7f7f08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:08.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.581+0000 7f7efffff640 1 -- 172.21.15.12:0/1939104138 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7f18143800 con 0x7f7f181534c0 2024-04-27T07:04:08.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.741+0000 7f7f1de09640 1 -- 172.21.15.12:0/1939104138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f18150a40 con 0x7f7f181534c0 2024-04-27T07:04:08.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7efffff640 1 -- 172.21.15.12:0/1939104138 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10849 (secure 0 0 0) 0x7f7f0c057390 con 0x7f7f181534c0 2024-04-27T07:04:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:08.749 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:08.349154+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:08.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 -- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7f040682f0 msgr2=0x7f7f0406a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7f040682f0 0x7f7f0406a7b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7f08000c10 tx=0x7f7f08009040 comp rx=0 tx=0).stop 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 -- 172.21.15.12:0/1939104138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f181534c0 msgr2=0x7f7f18142e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f181534c0 0x7f7f18142e40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00f4f0 tx=0x7f7f0c00f9b0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 -- 172.21.15.12:0/1939104138 shutdown_connections 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7f040682f0 0x7f7f0406a7b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7f18143380 0x7f7f181441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f181534c0 0x7f7f18142e40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 --2- 172.21.15.12:0/1939104138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7f00004da0 0x7f7f18152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 -- 172.21.15.12:0/1939104138 >> 172.21.15.12:0/1939104138 conn(0x7f7f18073c00 msgr2=0x7f7f181464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 -- 172.21.15.12:0/1939104138 shutdown_connections 2024-04-27T07:04:08.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.745+0000 7f7f1de09640 1 -- 172.21.15.12:0/1939104138 wait complete. 2024-04-27T07:04:08.768 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-27T07:04:08.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:04:08.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc8010b3640 1 Processor -- start 2024-04-27T07:04:08.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc8010b3640 1 -- start start 2024-04-27T07:04:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc8010b3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc7fc067b70 0x7fc7fc0aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc8010b3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7fc0b0ba0 con 0x7fc7fc0b0150 2024-04-27T07:04:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc8010b3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7fc067f50 con 0x7fc7fc1483f0 2024-04-27T07:04:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc8010b3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fc7fc0b05e0 con 0x7fc7fc067b70 2024-04-27T07:04:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7fad9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc7fc067b70 0x7fc7fc0aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7fa59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc7fc0b0150 0x7fc7fc13e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34748/0 (socket says 172.21.15.12:34748) 2024-04-27T07:04:08.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7fa59c640 1 -- 172.21.15.12:0/804498821 learned_addr learned my addr 172.21.15.12:0/804498821 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:08.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7fad9d640 1 -- 172.21.15.12:0/804498821 >> v1:172.21.15.53:6789/0 conn(0x7fc7fc1483f0 legacy=0x7fc7fc148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:08.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7fad9d640 1 -- 172.21.15.12:0/804498821 >> v1:172.21.15.12:6789/0 conn(0x7fc7fc0b0150 legacy=0x7fc7fc13e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7fad9d640 1 -- 172.21.15.12:0/804498821 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7fc0b08c0 con 0x7fc7fc067b70 2024-04-27T07:04:08.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7f9d9b640 1 -- 172.21.15.12:0/804498821 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902454326 0 0) 0x7fc7fc0b0ba0 con 0x7fc7fc0b0150 2024-04-27T07:04:08.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7fad9d640 1 --2- 172.21.15.12:0/804498821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc067b70 0x7fc7fc0aedc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009d90 tx=0x7fc7f401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37386245d0be741 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:08.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7f9d9b640 1 -- 172.21.15.12:0/804498821 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40b2070 con 0x7fc7fc067b70 2024-04-27T07:04:08.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7f9d9b640 1 -- 172.21.15.12:0/804498821 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f40a5870 con 0x7fc7fc067b70 2024-04-27T07:04:08.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.877+0000 7fc7f9d9b640 1 -- 172.21.15.12:0/804498821 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40a5c40 con 0x7fc7fc067b70 2024-04-27T07:04:08.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- 172.21.15.12:0/804498821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc067b70 msgr2=0x7fc7fc0aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 --2- 172.21.15.12:0/804498821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc067b70 0x7fc7fc0aedc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009d90 tx=0x7fc7f401c8a0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- 172.21.15.12:0/804498821 shutdown_connections 2024-04-27T07:04:08.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 --2- 172.21.15.12:0/804498821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc067b70 0x7fc7fc0aedc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- 172.21.15.12:0/804498821 >> 172.21.15.12:0/804498821 conn(0x7fc7fc0b65d0 msgr2=0x7fc7fc0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:08.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- 172.21.15.12:0/804498821 shutdown_connections 2024-04-27T07:04:08.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- 172.21.15.12:0/804498821 wait complete. 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 Processor -- start 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- start start 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7fc067b70 0x7fc7fc0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc1483f0 0x7fc7fc1de430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7fc0b2e70 0x7fc7fc1de970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc152dc0 con 0x7fc7fc1483f0 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc152f40 con 0x7fc7fc0b2e70 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc7fc1530c0 con 0x7fc7fc067b70 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc1483f0 0x7fc7fc1de430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc1483f0 0x7fc7fc1de430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38248/0 (socket says 172.21.15.12:38248) 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fa59c640 1 -- 172.21.15.12:0/2344244787 learned_addr learned my addr 172.21.15.12:0/2344244787 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fb59e640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7fc0b2e70 0x7fc7fc1de970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fad9d640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7fc067b70 0x7fc7fc0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fa59c640 1 -- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7fc067b70 msgr2=0x7fc7fc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fa59c640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7fc067b70 0x7fc7fc0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fa59c640 1 -- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7fc0b2e70 msgr2=0x7fc7fc1de970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fa59c640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7fc0b2e70 0x7fc7fc1de970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fa59c640 1 -- 172.21.15.12:0/2344244787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7fc0b1cb0 con 0x7fc7fc1483f0 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fb59e640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7fc0b2e70 0x7fc7fc1de970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7fa59c640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc1483f0 0x7fc7fc1de430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400c550 tx=0x7fc7e400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7f37fe640 1 -- 172.21.15.12:0/2344244787 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4017070 con 0x7fc7fc1483f0 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7f37fe640 1 -- 172.21.15.12:0/2344244787 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e4005d10 con 0x7fc7fc1483f0 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- 172.21.15.12:0/2344244787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7fc13fba0 con 0x7fc7fc1483f0 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc7f37fe640 1 -- 172.21.15.12:0/2344244787 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4007330 con 0x7fc7fc1483f0 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- 172.21.15.12:0/2344244787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7fc0b2230 con 0x7fc7fc1483f0 2024-04-27T07:04:08.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.881+0000 7fc8010b3640 1 -- 172.21.15.12:0/2344244787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7fc0b1110 con 0x7fc7fc1483f0 2024-04-27T07:04:08.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.885+0000 7fc7f37fe640 1 -- 172.21.15.12:0/2344244787 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7fc7e401a070 con 0x7fc7fc1483f0 2024-04-27T07:04:08.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.885+0000 7fc7f37fe640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc7dc0682f0 0x7fc7dc06a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:08.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.885+0000 7fc7f37fe640 1 -- 172.21.15.12:0/2344244787 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fc7e40857e0 con 0x7fc7fc1483f0 2024-04-27T07:04:08.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.885+0000 7fc7fad9d640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc7dc0682f0 0x7fc7dc06a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:08.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.885+0000 7fc7fad9d640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc7dc0682f0 0x7fc7dc06a7b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40af000 tx=0x7fc7f40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:08.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:08.889+0000 7fc7f37fe640 1 -- 172.21.15.12:0/2344244787 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc7fc0b1110 con 0x7fc7fc1483f0 2024-04-27T07:04:09.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.077+0000 7fc8010b3640 1 -- 172.21.15.12:0/2344244787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7fc069330 con 0x7fc7fc1483f0 2024-04-27T07:04:09.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc7f37fe640 1 -- 172.21.15.12:0/2344244787 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7fc7e40563e0 con 0x7fc7fc1483f0 2024-04-27T07:04:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:09.085 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6833/4173401303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4173401303},{"type":"v1","addr":"172.21.15.12:6833","nonce":4173401303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-27T07:04:09.085 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 2 2024-04-27T07:04:09.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 -- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc7dc0682f0 msgr2=0x7fc7dc06a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc7dc0682f0 0x7fc7dc06a7b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40af000 tx=0x7fc7f40a7070 comp rx=0 tx=0).stop 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 -- 172.21.15.12:0/2344244787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc1483f0 msgr2=0x7fc7fc1de430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc1483f0 0x7fc7fc1de430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400c550 tx=0x7fc7e400ca10 comp rx=0 tx=0).stop 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 -- 172.21.15.12:0/2344244787 shutdown_connections 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc7dc0682f0 0x7fc7dc06a7b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc7fc0b2e70 0x7fc7fc1de970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7fc1483f0 0x7fc7fc1de430 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 --2- 172.21.15.12:0/2344244787 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc7fc067b70 0x7fc7fc0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 -- 172.21.15.12:0/2344244787 >> 172.21.15.12:0/2344244787 conn(0x7fc7fc0b65d0 msgr2=0x7fc7fc0b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 -- 172.21.15.12:0/2344244787 shutdown_connections 2024-04-27T07:04:09.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:09.081+0000 7fc8010b3640 1 -- 172.21.15.12:0/2344244787 wait complete. 2024-04-27T07:04:11.102 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:04:11.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6b5153640 1 Processor -- start 2024-04-27T07:04:11.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6b5153640 1 -- start start 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6b5153640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b00b8b10 0x7fe6b0146450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6b5153640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe6b0146990 0x7fe6b013f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6b5153640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b006e520 con 0x7fe6b00b8730 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6b5153640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fe6b0145c70 con 0x7fe6b0146990 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6b5153640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b00b8f70 con 0x7fe6b00b8b10 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6aed9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe6b00b8730 0x7fe6b01496f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34754/0 (socket says 172.21.15.12:34754) 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6aed9d640 1 -- 172.21.15.12:0/1494275362 learned_addr learned my addr 172.21.15.12:0/1494275362 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6af59e640 1 --2- 172.21.15.12:0/1494275362 >> v2:172.21.15.12:3300/0 conn(0x7fe6b0146990 0x7fe6b013f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6ae59c640 1 --2- 172.21.15.12:0/1494275362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b00b8b10 0x7fe6b0146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6add9b640 1 -- 172.21.15.12:0/1494275362 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4236932741 0 0) 0x7fe6a8001120 con 0x7fe6b00b8730 2024-04-27T07:04:11.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6add9b640 1 -- 172.21.15.12:0/1494275362 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe69c003530 con 0x7fe6b00b8730 2024-04-27T07:04:11.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6af59e640 1 -- 172.21.15.12:0/1494275362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b00b8b10 msgr2=0x7fe6b0146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:11.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6af59e640 1 --2- 172.21.15.12:0/1494275362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b00b8b10 0x7fe6b0146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6af59e640 1 -- 172.21.15.12:0/1494275362 >> v1:172.21.15.12:6789/0 conn(0x7fe6b00b8730 legacy=0x7fe6b01496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.221+0000 7fe6af59e640 1 -- 172.21.15.12:0/1494275362 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b0149fb0 con 0x7fe6b0146990 2024-04-27T07:04:11.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6af59e640 1 --2- 172.21.15.12:0/1494275362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b0146990 0x7fe6b013f940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009d90 tx=0x7fe6a001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342e9ea7f0aa8d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6add9b640 1 -- 172.21.15.12:0/1494275362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00b2070 con 0x7fe6b0146990 2024-04-27T07:04:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6add9b640 1 -- 172.21.15.12:0/1494275362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a00a5820 con 0x7fe6b0146990 2024-04-27T07:04:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6add9b640 1 -- 172.21.15.12:0/1494275362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00a5bf0 con 0x7fe6b0146990 2024-04-27T07:04:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 -- 172.21.15.12:0/1494275362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b0146990 msgr2=0x7fe6b013f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 --2- 172.21.15.12:0/1494275362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b0146990 0x7fe6b013f940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009d90 tx=0x7fe6a001c850 comp rx=0 tx=0).stop 2024-04-27T07:04:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 -- 172.21.15.12:0/1494275362 shutdown_connections 2024-04-27T07:04:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 --2- 172.21.15.12:0/1494275362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b0146990 0x7fe6b013f940 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 --2- 172.21.15.12:0/1494275362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b00b8b10 0x7fe6b0146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 -- 172.21.15.12:0/1494275362 >> 172.21.15.12:0/1494275362 conn(0x7fe6b009c480 msgr2=0x7fe6b009c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:11.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 -- 172.21.15.12:0/1494275362 shutdown_connections 2024-04-27T07:04:11.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 -- 172.21.15.12:0/1494275362 wait complete. 2024-04-27T07:04:11.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 Processor -- start 2024-04-27T07:04:11.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 -- start start 2024-04-27T07:04:11.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b00b8730 0x7fe6b0073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b00b8b10 0x7fe6b009b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b0146990 0x7fe6b0072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0153180 con 0x7fe6b00b8b10 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0153000 con 0x7fe6b00b8730 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6b5153640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b0153300 con 0x7fe6b0146990 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6ae59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b00b8b10 0x7fe6b009b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6ae59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b00b8b10 0x7fe6b009b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38274/0 (socket says 172.21.15.12:38274) 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.225+0000 7fe6ae59c640 1 -- 172.21.15.12:0/38719297 learned_addr learned my addr 172.21.15.12:0/38719297 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6aed9d640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b00b8730 0x7fe6b0073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6af59e640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b0146990 0x7fe6b0072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6ae59c640 1 -- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b0146990 msgr2=0x7fe6b0072ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6ae59c640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b0146990 0x7fe6b0072ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6ae59c640 1 -- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b00b8730 msgr2=0x7fe6b0073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6ae59c640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b00b8730 0x7fe6b0073690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6ae59c640 1 -- 172.21.15.12:0/38719297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b009be80 con 0x7fe6b00b8b10 2024-04-27T07:04:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6af59e640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b0146990 0x7fe6b0072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6ae59c640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b00b8b10 0x7fe6b009b340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe69800e960 tx=0x7fe69800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6a77fe640 1 -- 172.21.15.12:0/38719297 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698017070 con 0x7fe6b00b8b10 2024-04-27T07:04:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6a77fe640 1 -- 172.21.15.12:0/38719297 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe69800ca00 con 0x7fe6b00b8b10 2024-04-27T07:04:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6b5153640 1 -- 172.21.15.12:0/38719297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b0073fd0 con 0x7fe6b00b8b10 2024-04-27T07:04:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6b5153640 1 -- 172.21.15.12:0/38719297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b0073bd0 con 0x7fe6b00b8b10 2024-04-27T07:04:11.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6a77fe640 1 -- 172.21.15.12:0/38719297 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69800cdb0 con 0x7fe6b00b8b10 2024-04-27T07:04:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6b5153640 1 -- 172.21.15.12:0/38719297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b0069330 con 0x7fe6b00b8b10 2024-04-27T07:04:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6a77fe640 1 -- 172.21.15.12:0/38719297 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7fe69801a070 con 0x7fe6b00b8b10 2024-04-27T07:04:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6a77fe640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6900682f0 0x7fe69006a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6a77fe640 1 -- 172.21.15.12:0/38719297 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7fe698086d20 con 0x7fe6b00b8b10 2024-04-27T07:04:11.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6aed9d640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6900682f0 0x7fe69006a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.229+0000 7fe6aed9d640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6900682f0 0x7fe69006a7b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8005890 tx=0x7fe6a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:11.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.233+0000 7fe6a77fe640 1 -- 172.21.15.12:0/38719297 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe698057780 con 0x7fe6b00b8b10 2024-04-27T07:04:11.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.437+0000 7fe6b5153640 1 -- 172.21.15.12:0/38719297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6b00a07b0 con 0x7fe6b00b8b10 2024-04-27T07:04:11.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6a77fe640 1 -- 172.21.15.12:0/38719297 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2216 (secure 0 0 0) 0x7fe698057080 con 0x7fe6b00b8b10 2024-04-27T07:04:11.452 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:11.452 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1908224930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1908224930},{"type":"v1","addr":"172.21.15.53:6835","nonce":1908224930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6833/4173401303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4173401303},{"type":"v1","addr":"172.21.15.12:6833","nonce":4173401303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-27T07:04:11.453 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2024-04-27T07:04:11.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 -- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6900682f0 msgr2=0x7fe69006a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6900682f0 0x7fe69006a7b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8005890 tx=0x7fe6a8009040 comp rx=0 tx=0).stop 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 -- 172.21.15.12:0/38719297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b00b8b10 msgr2=0x7fe6b009b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b00b8b10 0x7fe6b009b340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe69800e960 tx=0x7fe69800ee20 comp rx=0 tx=0).stop 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 -- 172.21.15.12:0/38719297 shutdown_connections 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6900682f0 0x7fe69006a7b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6b0146990 0x7fe6b0072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6b00b8b10 0x7fe6b009b340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 --2- 172.21.15.12:0/38719297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6b00b8730 0x7fe6b0073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 -- 172.21.15.12:0/38719297 >> 172.21.15.12:0/38719297 conn(0x7fe6b009c480 msgr2=0x7fe6b013dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 -- 172.21.15.12:0/38719297 shutdown_connections 2024-04-27T07:04:11.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.449+0000 7fe6b5153640 1 -- 172.21.15.12:0/38719297 wait complete. 2024-04-27T07:04:11.472 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:11.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a2b6b640 1 Processor -- start 2024-04-27T07:04:11.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a2b6b640 1 -- start start 2024-04-27T07:04:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a2b6b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f879c14f410 0x7f879c151800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a2b6b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879c06dac0 con 0x7f879c0b9a10 2024-04-27T07:04:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a2b6b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879c06e890 con 0x7f879c0b3680 2024-04-27T07:04:11.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a2b6b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f879c0b6cd0 con 0x7f879c14f410 2024-04-27T07:04:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a0907640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f879c0b9a10 0x7f879c0b65c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34770/0 (socket says 172.21.15.12:34770) 2024-04-27T07:04:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a0907640 1 -- 172.21.15.12:0/1575700241 learned_addr learned my addr 172.21.15.12:0/1575700241 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a1108640 1 --2- 172.21.15.12:0/1575700241 >> v2:172.21.15.12:3300/0 conn(0x7f879c14f410 0x7f879c151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f879b7fe640 1 -- 172.21.15.12:0/1575700241 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3128129138 0 0) 0x7f87840014a0 con 0x7f879c0b9a10 2024-04-27T07:04:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f879b7fe640 1 -- 172.21.15.12:0/1575700241 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f877c003530 con 0x7f879c0b9a10 2024-04-27T07:04:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f879b7fe640 1 -- 172.21.15.12:0/1575700241 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1589358164 0 0) 0x7f879c06e890 con 0x7f879c0b3680 2024-04-27T07:04:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a1108640 1 -- 172.21.15.12:0/1575700241 >> v1:172.21.15.53:6789/0 conn(0x7f879c0b3680 legacy=0x7f879c145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a1108640 1 -- 172.21.15.12:0/1575700241 >> v1:172.21.15.12:6789/0 conn(0x7f879c0b9a10 legacy=0x7f879c0b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.593+0000 7f87a1108640 1 -- 172.21.15.12:0/1575700241 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c0b9df0 con 0x7f879c14f410 2024-04-27T07:04:11.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a1108640 1 --2- 172.21.15.12:0/1575700241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c14f410 0x7f879c151800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87940097c0 tx=0x7f879401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59f761c798b70d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:11.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f879b7fe640 1 -- 172.21.15.12:0/1575700241 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940b0070 con 0x7f879c14f410 2024-04-27T07:04:11.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f879b7fe640 1 -- 172.21.15.12:0/1575700241 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87940ab600 con 0x7f879c14f410 2024-04-27T07:04:11.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f879b7fe640 1 -- 172.21.15.12:0/1575700241 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87940ab9d0 con 0x7f879c14f410 2024-04-27T07:04:11.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- 172.21.15.12:0/1575700241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c14f410 msgr2=0x7f879c151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 --2- 172.21.15.12:0/1575700241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c14f410 0x7f879c151800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87940097c0 tx=0x7f879401c410 comp rx=0 tx=0).stop 2024-04-27T07:04:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- 172.21.15.12:0/1575700241 shutdown_connections 2024-04-27T07:04:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 --2- 172.21.15.12:0/1575700241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c14f410 0x7f879c151800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- 172.21.15.12:0/1575700241 >> 172.21.15.12:0/1575700241 conn(0x7f879c067d10 msgr2=0x7f879c068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- 172.21.15.12:0/1575700241 shutdown_connections 2024-04-27T07:04:11.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- 172.21.15.12:0/1575700241 wait complete. 2024-04-27T07:04:11.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 Processor -- start 2024-04-27T07:04:11.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- start start 2024-04-27T07:04:11.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c0b3680 0x7f879c0b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f879c0b9a10 0x7f879c1e3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c14f410 0x7f879c1e3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f879c06d480 con 0x7f879c0b3680 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f879c06d300 con 0x7f879c14f410 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f879c073ea0 con 0x7f879c0b9a10 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a0907640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c0b3680 0x7f879c0b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a0907640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c0b3680 0x7f879c0b7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38294/0 (socket says 172.21.15.12:38294) 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a0907640 1 -- 172.21.15.12:0/3661721611 learned_addr learned my addr 172.21.15.12:0/3661721611 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f879bfff640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f879c0b9a10 0x7f879c1e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a1108640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c14f410 0x7f879c1e3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a0907640 1 -- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f879c0b9a10 msgr2=0x7f879c1e3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a0907640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f879c0b9a10 0x7f879c1e3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a0907640 1 -- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c14f410 msgr2=0x7f879c1e3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a0907640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c14f410 0x7f879c1e3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a0907640 1 -- 172.21.15.12:0/3661721611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c145080 con 0x7f879c0b3680 2024-04-27T07:04:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a0907640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c0b3680 0x7f879c0b7a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f878400cb80 tx=0x7f878400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87997fa640 1 -- 172.21.15.12:0/3661721611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784017070 con 0x7f879c0b3680 2024-04-27T07:04:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87997fa640 1 -- 172.21.15.12:0/3661721611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878400ad40 con 0x7f879c0b3680 2024-04-27T07:04:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87997fa640 1 -- 172.21.15.12:0/3661721611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784012640 con 0x7f879c0b3680 2024-04-27T07:04:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- 172.21.15.12:0/3661721611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879c13d810 con 0x7f879c0b3680 2024-04-27T07:04:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.597+0000 7f87a2b6b640 1 -- 172.21.15.12:0/3661721611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879c0af5a0 con 0x7f879c0b3680 2024-04-27T07:04:11.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.601+0000 7f87a2b6b640 1 -- 172.21.15.12:0/3661721611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879c068c20 con 0x7f879c0b3680 2024-04-27T07:04:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.601+0000 7f87997fa640 1 -- 172.21.15.12:0/3661721611 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7f878401a070 con 0x7f879c0b3680 2024-04-27T07:04:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.601+0000 7f87997fa640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8780068320 0x7f878006a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.601+0000 7f87997fa640 1 -- 172.21.15.12:0/3661721611 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f87840863a0 con 0x7f879c0b3680 2024-04-27T07:04:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.601+0000 7f879bfff640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8780068320 0x7f878006a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.601+0000 7f879bfff640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8780068320 0x7f878006a7e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f879c09e650 tx=0x7f878c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:11.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.605+0000 7f87997fa640 1 -- 172.21.15.12:0/3661721611 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f879c06dac0 con 0x7f879c0b3680 2024-04-27T07:04:11.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.769+0000 7f87a2b6b640 1 -- 172.21.15.12:0/3661721611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f879c0b86b0 con 0x7f879c0b3680 2024-04-27T07:04:11.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87997fa640 1 -- 172.21.15.12:0/3661721611 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12824 (secure 0 0 0) 0x7f8784056d80 con 0x7f879c0b3680 2024-04-27T07:04:11.777 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:11.777 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:11.546110+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 -- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8780068320 msgr2=0x7f878006a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8780068320 0x7f878006a7e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f879c09e650 tx=0x7f878c009040 comp rx=0 tx=0).stop 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 -- 172.21.15.12:0/3661721611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c0b3680 msgr2=0x7f879c0b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c0b3680 0x7f879c0b7a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f878400cb80 tx=0x7f878400a430 comp rx=0 tx=0).stop 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 -- 172.21.15.12:0/3661721611 shutdown_connections 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8780068320 0x7f878006a7e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c14f410 0x7f879c1e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f879c0b9a10 0x7f879c1e3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 --2- 172.21.15.12:0/3661721611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f879c0b3680 0x7f879c0b7a90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 -- 172.21.15.12:0/3661721611 >> 172.21.15.12:0/3661721611 conn(0x7f879c067d10 msgr2=0x7f879c14e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 -- 172.21.15.12:0/3661721611 shutdown_connections 2024-04-27T07:04:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.773+0000 7f87a2b6b640 1 -- 172.21.15.12:0/3661721611 wait complete. 2024-04-27T07:04:11.796 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:11.796 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:11.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886f26e640 1 Processor -- start 2024-04-27T07:04:11.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886f26e640 1 -- start start 2024-04-27T07:04:11.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886f26e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f88681489c0 0x7f88680b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886f26e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8868142010 0x7f88680b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886f26e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88680b6860 con 0x7f88680b6480 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886f26e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f88680b84e0 con 0x7f88681489c0 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886f26e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8868147ee0 con 0x7f8868142010 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886c809640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f88681489c0 0x7f88680b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886d00a640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f88680b6480 0x7f8868141900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34772/0 (socket says 172.21.15.12:34772) 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886d00a640 1 -- 172.21.15.12:0/1587183909 learned_addr learned my addr 172.21.15.12:0/1587183909 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886d80b640 1 --2- 172.21.15.12:0/1587183909 >> v2:172.21.15.53:3300/0 conn(0x7f8868142010 0x7f88680b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886c809640 1 -- 172.21.15.12:0/1587183909 >> v2:172.21.15.53:3300/0 conn(0x7f8868142010 msgr2=0x7f88680b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886c809640 1 --2- 172.21.15.12:0/1587183909 >> v2:172.21.15.53:3300/0 conn(0x7f8868142010 0x7f88680b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f885ffff640 1 -- 172.21.15.12:0/1587183909 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 935548238 0 0) 0x7f88680b6860 con 0x7f88680b6480 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886c809640 1 -- 172.21.15.12:0/1587183909 >> v1:172.21.15.12:6789/0 conn(0x7f88680b6480 legacy=0x7f8868141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886c809640 1 -- 172.21.15.12:0/1587183909 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88681481c0 con 0x7f88681489c0 2024-04-27T07:04:11.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.921+0000 7f886d80b640 1 --2- 172.21.15.12:0/1587183909 >> v2:172.21.15.53:3300/0 conn(0x7f8868142010 0x7f88680b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:11.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886c809640 1 --2- 172.21.15.12:0/1587183909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88681489c0 0x7f88680b5ae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88680b55b0 tx=0x7f885801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301b5666e51e636b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:11.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f885ffff640 1 -- 172.21.15.12:0/1587183909 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88580b0070 con 0x7f88681489c0 2024-04-27T07:04:11.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f885ffff640 1 -- 172.21.15.12:0/1587183909 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88580ab600 con 0x7f88681489c0 2024-04-27T07:04:11.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f885ffff640 1 -- 172.21.15.12:0/1587183909 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88580ab9d0 con 0x7f88681489c0 2024-04-27T07:04:11.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 -- 172.21.15.12:0/1587183909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88681489c0 msgr2=0x7f88680b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 --2- 172.21.15.12:0/1587183909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88681489c0 0x7f88680b5ae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88680b55b0 tx=0x7f885801c3f0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 -- 172.21.15.12:0/1587183909 shutdown_connections 2024-04-27T07:04:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 --2- 172.21.15.12:0/1587183909 >> v2:172.21.15.53:3300/0 conn(0x7f8868142010 0x7f88680b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 --2- 172.21.15.12:0/1587183909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88681489c0 0x7f88680b5ae0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 -- 172.21.15.12:0/1587183909 >> 172.21.15.12:0/1587183909 conn(0x7f88680bd620 msgr2=0x7f88680bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 -- 172.21.15.12:0/1587183909 shutdown_connections 2024-04-27T07:04:11.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 -- 172.21.15.12:0/1587183909 wait complete. 2024-04-27T07:04:11.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 Processor -- start 2024-04-27T07:04:11.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 -- start start 2024-04-27T07:04:11.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88680b6480 0x7f88681e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8868142010 0x7f88681e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88681489c0 0x7f8868135360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8868154e70 con 0x7f8868142010 2024-04-27T07:04:11.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8868142420 con 0x7f88681489c0 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886f26e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8868154ff0 con 0x7f88680b6480 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886c809640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8868142010 0x7f88681e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886c809640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8868142010 0x7f88681e4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38308/0 (socket says 172.21.15.12:38308) 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886c809640 1 -- 172.21.15.12:0/2929689091 learned_addr learned my addr 172.21.15.12:0/2929689091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886d80b640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88681489c0 0x7f8868135360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886d00a640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88680b6480 0x7f88681e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886c809640 1 -- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88680b6480 msgr2=0x7f88681e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886c809640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88680b6480 0x7f88681e3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886c809640 1 -- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88681489c0 msgr2=0x7f8868135360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886c809640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88681489c0 0x7f8868135360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886c809640 1 -- 172.21.15.12:0/2929689091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8868144020 con 0x7f8868142010 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886d80b640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88681489c0 0x7f8868135360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.925+0000 7f886c809640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8868142010 0x7f88681e4100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88580ae040 tx=0x7f8858019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:11.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f885dffb640 1 -- 172.21.15.12:0/2929689091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88580b8020 con 0x7f8868142010 2024-04-27T07:04:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f885dffb640 1 -- 172.21.15.12:0/2929689091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88580b4630 con 0x7f8868142010 2024-04-27T07:04:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f886f26e640 1 -- 172.21.15.12:0/2929689091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886813bfc0 con 0x7f8868142010 2024-04-27T07:04:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f885dffb640 1 -- 172.21.15.12:0/2929689091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88580b0040 con 0x7f8868142010 2024-04-27T07:04:11.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f886f26e640 1 -- 172.21.15.12:0/2929689091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f886813a5f0 con 0x7f8868142010 2024-04-27T07:04:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f886f26e640 1 -- 172.21.15.12:0/2929689091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88681443c0 con 0x7f8868142010 2024-04-27T07:04:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f885dffb640 1 -- 172.21.15.12:0/2929689091 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7f88580bb020 con 0x7f8868142010 2024-04-27T07:04:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f885dffb640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f884c0682f0 0x7f884c06a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f885dffb640 1 -- 172.21.15.12:0/2929689091 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f88581202b0 con 0x7f8868142010 2024-04-27T07:04:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f886d00a640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f884c0682f0 0x7f884c06a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:11.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.929+0000 7f886d00a640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f884c0682f0 0x7f884c06a7b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8864000f70 tx=0x7f8864009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:11.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:11.933+0000 7f885dffb640 1 -- 172.21.15.12:0/2929689091 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f88681443c0 con 0x7f8868142010 2024-04-27T07:04:12.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 -- 172.21.15.12:0/2929689091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8868068c20 con 0x7f8868142010 2024-04-27T07:04:12.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f885dffb640 1 -- 172.21.15.12:0/2929689091 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f88580c2030 con 0x7f8868142010 2024-04-27T07:04:12.101 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 -- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f884c0682f0 msgr2=0x7f884c06a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f884c0682f0 0x7f884c06a7b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8864000f70 tx=0x7f8864009040 comp rx=0 tx=0).stop 2024-04-27T07:04:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 -- 172.21.15.12:0/2929689091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8868142010 msgr2=0x7f88681e4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8868142010 0x7f88681e4100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88580ae040 tx=0x7f8858019190 comp rx=0 tx=0).stop 2024-04-27T07:04:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 -- 172.21.15.12:0/2929689091 shutdown_connections 2024-04-27T07:04:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f884c0682f0 0x7f884c06a7b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88681489c0 0x7f8868135360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8868142010 0x7f88681e4100 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 --2- 172.21.15.12:0/2929689091 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88680b6480 0x7f88681e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 -- 172.21.15.12:0/2929689091 >> 172.21.15.12:0/2929689091 conn(0x7f88680bd620 msgr2=0x7f8868140540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 -- 172.21.15.12:0/2929689091 shutdown_connections 2024-04-27T07:04:12.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.097+0000 7f886f26e640 1 -- 172.21.15.12:0/2929689091 wait complete. 2024-04-27T07:04:12.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-27T07:04:12.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4bd380640 1 Processor -- start 2024-04-27T07:04:12.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4bd380640 1 -- start start 2024-04-27T07:04:12.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4bd380640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd4b80b9a10 0x7fd4b80b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4bd380640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b80b3680 0x7fd4b8145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4bd380640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd4b81460b0 0x7fd4b81490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4bd380640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fd4b806dac0 con 0x7fd4b81460b0 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4bd380640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd4b806e890 con 0x7fd4b80b9a10 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4bd380640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b80b9df0 con 0x7fd4b80b3680 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4b77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd4b81460b0 0x7fd4b81490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4b77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd4b81460b0 0x7fd4b81490e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38320/0 (socket says 172.21.15.12:38320) 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4b77fe640 1 -- 172.21.15.12:0/1092561214 learned_addr learned my addr 172.21.15.12:0/1092561214 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4b6ffd640 1 --2- 172.21.15.12:0/1092561214 >> v2:172.21.15.53:3300/0 conn(0x7fd4b80b9a10 0x7fd4b80b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4b67fc640 1 --2- 172.21.15.12:0/1092561214 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b80b3680 0x7fd4b8145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4b77fe640 1 -- 172.21.15.12:0/1092561214 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b80b3680 msgr2=0x7fd4b8145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4b77fe640 1 --2- 172.21.15.12:0/1092561214 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b80b3680 0x7fd4b8145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4b77fe640 1 -- 172.21.15.12:0/1092561214 >> v2:172.21.15.53:3300/0 conn(0x7fd4b80b9a10 msgr2=0x7fd4b80b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4b77fe640 1 --2- 172.21.15.12:0/1092561214 >> v2:172.21.15.53:3300/0 conn(0x7fd4b80b9a10 0x7fd4b80b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.237+0000 7fd4b77fe640 1 -- 172.21.15.12:0/1092561214 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b806e090 con 0x7fd4b81460b0 2024-04-27T07:04:12.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4b77fe640 1 --2- 172.21.15.12:0/1092561214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b81460b0 0x7fd4b81490e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4b806ed00 tx=0x7fd4b001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb1bcc01efa31f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:12.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4b5ffb640 1 -- 172.21.15.12:0/1092561214 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00b0070 con 0x7fd4b81460b0 2024-04-27T07:04:12.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4b5ffb640 1 -- 172.21.15.12:0/1092561214 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b00ab600 con 0x7fd4b81460b0 2024-04-27T07:04:12.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4b5ffb640 1 -- 172.21.15.12:0/1092561214 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00ab9d0 con 0x7fd4b81460b0 2024-04-27T07:04:12.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 -- 172.21.15.12:0/1092561214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b81460b0 msgr2=0x7fd4b81490e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 --2- 172.21.15.12:0/1092561214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b81460b0 0x7fd4b81490e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4b806ed00 tx=0x7fd4b001c1d0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 -- 172.21.15.12:0/1092561214 shutdown_connections 2024-04-27T07:04:12.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 --2- 172.21.15.12:0/1092561214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b81460b0 0x7fd4b81490e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 --2- 172.21.15.12:0/1092561214 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b80b3680 0x7fd4b8145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 --2- 172.21.15.12:0/1092561214 >> v2:172.21.15.53:3300/0 conn(0x7fd4b80b9a10 0x7fd4b80b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 -- 172.21.15.12:0/1092561214 >> 172.21.15.12:0/1092561214 conn(0x7fd4b8067d10 msgr2=0x7fd4b8068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:12.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 -- 172.21.15.12:0/1092561214 shutdown_connections 2024-04-27T07:04:12.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 -- 172.21.15.12:0/1092561214 wait complete. 2024-04-27T07:04:12.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 Processor -- start 2024-04-27T07:04:12.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.241+0000 7fd4bd380640 1 -- start start 2024-04-27T07:04:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4bd380640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b80b3680 0x7fd4b81e0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4bd380640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b80b9a10 0x7fd4b809d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4bd380640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b81460b0 0x7fd4b81e47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4bd380640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b80b6b00 con 0x7fd4b80b3680 2024-04-27T07:04:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4bd380640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b80740b0 con 0x7fd4b80b9a10 2024-04-27T07:04:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4bd380640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b80b6c80 con 0x7fd4b81460b0 2024-04-27T07:04:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b80b3680 0x7fd4b81e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b80b3680 0x7fd4b81e0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38324/0 (socket says 172.21.15.12:38324) 2024-04-27T07:04:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b6ffd640 1 -- 172.21.15.12:0/1312232784 learned_addr learned my addr 172.21.15.12:0/1312232784 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b67fc640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b80b9a10 0x7fd4b809d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b77fe640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b81460b0 0x7fd4b81e47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b6ffd640 1 -- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b81460b0 msgr2=0x7fd4b81e47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b6ffd640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b81460b0 0x7fd4b81e47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b6ffd640 1 -- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b80b9a10 msgr2=0x7fd4b809d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b6ffd640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b80b9a10 0x7fd4b809d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b6ffd640 1 -- 172.21.15.12:0/1312232784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b81e3fe0 con 0x7fd4b80b3680 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b77fe640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b81460b0 0x7fd4b81e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b6ffd640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b80b3680 0x7fd4b81e0820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8007ef0 tx=0x7fd4a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4af7fe640 1 -- 172.21.15.12:0/1312232784 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8014620 con 0x7fd4b80b3680 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4af7fe640 1 -- 172.21.15.12:0/1312232784 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a8019070 con 0x7fd4b80b3680 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4bd380640 1 -- 172.21.15.12:0/1312232784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b81e42c0 con 0x7fd4b80b3680 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4af7fe640 1 -- 172.21.15.12:0/1312232784 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8014a00 con 0x7fd4b80b3680 2024-04-27T07:04:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4bd380640 1 -- 172.21.15.12:0/1312232784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b8150110 con 0x7fd4b80b3680 2024-04-27T07:04:12.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4bd380640 1 -- 172.21.15.12:0/1312232784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b8068c20 con 0x7fd4b80b3680 2024-04-27T07:04:12.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4af7fe640 1 -- 172.21.15.12:0/1312232784 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7fd4a8014ba0 con 0x7fd4b80b3680 2024-04-27T07:04:12.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4af7fe640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd498068320 0x7fd49806a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4af7fe640 1 -- 172.21.15.12:0/1312232784 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd4a8086e50 con 0x7fd4b80b3680 2024-04-27T07:04:12.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.245+0000 7fd4b67fc640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd498068320 0x7fd49806a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.249+0000 7fd4b67fc640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd498068320 0x7fd49806a7e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4b809df30 tx=0x7fd4a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.249+0000 7fd4af7fe640 1 -- 172.21.15.12:0/1312232784 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4a80578d0 con 0x7fd4b80b3680 2024-04-27T07:04:12.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 -- 172.21.15.12:0/1312232784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd4b80b85f0 con 0x7fd4b80b3680 2024-04-27T07:04:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4af7fe640 1 -- 172.21.15.12:0/1312232784 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fd4a80571d0 con 0x7fd4b80b3680 2024-04-27T07:04:12.422 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:12.422 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2024-04-27T07:04:12.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:12.423 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2024-04-27T07:04:12.423 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2024-04-27T07:04:12.423 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:12.423 INFO:teuthology.orchestra.run.smithi012.stdout:] 2024-04-27T07:04:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 -- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd498068320 msgr2=0x7fd49806a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd498068320 0x7fd49806a7e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4b809df30 tx=0x7fd4a0008040 comp rx=0 tx=0).stop 2024-04-27T07:04:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 -- 172.21.15.12:0/1312232784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b80b3680 msgr2=0x7fd4b81e0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b80b3680 0x7fd4b81e0820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8007ef0 tx=0x7fd4a800e4b0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 -- 172.21.15.12:0/1312232784 shutdown_connections 2024-04-27T07:04:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd498068320 0x7fd49806a7e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4b81460b0 0x7fd4b81e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4b80b9a10 0x7fd4b809d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 --2- 172.21.15.12:0/1312232784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4b80b3680 0x7fd4b81e0820 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 -- 172.21.15.12:0/1312232784 >> 172.21.15.12:0/1312232784 conn(0x7fd4b8067d10 msgr2=0x7fd4b809b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.417+0000 7fd4bd380640 1 -- 172.21.15.12:0/1312232784 shutdown_connections 2024-04-27T07:04:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.421+0000 7fd4bd380640 1 -- 172.21.15.12:0/1312232784 wait complete. 2024-04-27T07:04:12.441 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-27T07:04:12.441 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-27T07:04:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a71c0640 1 Processor -- start 2024-04-27T07:04:12.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a71c0640 1 -- start start 2024-04-27T07:04:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a71c0640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb3a0067b70 0x7fb3a00b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a71c0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb3a00733e0 0x7fb3a0145100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a71c0640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3a0067f50 con 0x7fb3a0145640 2024-04-27T07:04:12.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a71c0640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a0145ab0 con 0x7fb3a0067b70 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a71c0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a00af590 con 0x7fb3a00733e0 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a4f5c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb3a0067b70 0x7fb3a00b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a575d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb3a0145640 0x7fb3a014b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34788/0 (socket says 172.21.15.12:34788) 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a575d640 1 -- 172.21.15.12:0/589488639 learned_addr learned my addr 172.21.15.12:0/589488639 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb39ffff640 1 --2- 172.21.15.12:0/589488639 >> v2:172.21.15.53:3300/0 conn(0x7fb3a00733e0 0x7fb3a0145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb39f7fe640 1 -- 172.21.15.12:0/589488639 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 955143128 0 0) 0x7fb3a0067f50 con 0x7fb3a0145640 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a4f5c640 1 -- 172.21.15.12:0/589488639 >> v2:172.21.15.53:3300/0 conn(0x7fb3a00733e0 msgr2=0x7fb3a0145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a4f5c640 1 --2- 172.21.15.12:0/589488639 >> v2:172.21.15.53:3300/0 conn(0x7fb3a00733e0 0x7fb3a0145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a4f5c640 1 -- 172.21.15.12:0/589488639 >> v1:172.21.15.12:6789/0 conn(0x7fb3a0145640 legacy=0x7fb3a014b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.557+0000 7fb3a4f5c640 1 -- 172.21.15.12:0/589488639 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a006f020 con 0x7fb3a0067b70 2024-04-27T07:04:12.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb3a4f5c640 1 --2- 172.21.15.12:0/589488639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0067b70 0x7fb3a00b0070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0074240 tx=0x7fb39801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceecb0beeee038f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb39f7fe640 1 -- 172.21.15.12:0/589488639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3980b2070 con 0x7fb3a0067b70 2024-04-27T07:04:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb39f7fe640 1 -- 172.21.15.12:0/589488639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3980a58e0 con 0x7fb3a0067b70 2024-04-27T07:04:12.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb39f7fe640 1 -- 172.21.15.12:0/589488639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3980a5cb0 con 0x7fb3a0067b70 2024-04-27T07:04:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb3a71c0640 1 -- 172.21.15.12:0/589488639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0067b70 msgr2=0x7fb3a00b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb3a71c0640 1 --2- 172.21.15.12:0/589488639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0067b70 0x7fb3a00b0070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0074240 tx=0x7fb39801c900 comp rx=0 tx=0).stop 2024-04-27T07:04:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb3a71c0640 1 -- 172.21.15.12:0/589488639 shutdown_connections 2024-04-27T07:04:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb3a71c0640 1 --2- 172.21.15.12:0/589488639 >> v2:172.21.15.53:3300/0 conn(0x7fb3a00733e0 0x7fb3a0145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb3a71c0640 1 --2- 172.21.15.12:0/589488639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a0067b70 0x7fb3a00b0070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb3a71c0640 1 -- 172.21.15.12:0/589488639 >> 172.21.15.12:0/589488639 conn(0x7fb3a006cac0 msgr2=0x7fb3a00b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb3a71c0640 1 -- 172.21.15.12:0/589488639 shutdown_connections 2024-04-27T07:04:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.561+0000 7fb3a71c0640 1 -- 172.21.15.12:0/589488639 wait complete. 2024-04-27T07:04:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 Processor -- start 2024-04-27T07:04:12.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 -- start start 2024-04-27T07:04:12.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a0067b70 0x7fb3a00720d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb3a0145640 0x7fb3a00b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a00b23e0 0x7fb3a01422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a006dc30 con 0x7fb3a00b23e0 2024-04-27T07:04:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a00b6420 con 0x7fb3a0067b70 2024-04-27T07:04:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a006ddb0 con 0x7fb3a0145640 2024-04-27T07:04:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a575d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a00b23e0 0x7fb3a01422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a575d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a00b23e0 0x7fb3a01422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38348/0 (socket says 172.21.15.12:38348) 2024-04-27T07:04:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a575d640 1 -- 172.21.15.12:0/3104946559 learned_addr learned my addr 172.21.15.12:0/3104946559 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a4f5c640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a0067b70 0x7fb3a00720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a575d640 1 -- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb3a0145640 msgr2=0x7fb3a00b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a575d640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb3a0145640 0x7fb3a00b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a575d640 1 -- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a0067b70 msgr2=0x7fb3a00720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a575d640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a0067b70 0x7fb3a00720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a575d640 1 -- 172.21.15.12:0/3104946559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a00b2940 con 0x7fb3a00b23e0 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a4f5c640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a0067b70 0x7fb3a00720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a575d640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a00b23e0 0x7fb3a01422d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb39000c3e0 tx=0x7fb39000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb39d7fa640 1 -- 172.21.15.12:0/3104946559 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3900074e0 con 0x7fb3a00b23e0 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb39d7fa640 1 -- 172.21.15.12:0/3104946559 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3900047f0 con 0x7fb3a00b23e0 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 -- 172.21.15.12:0/3104946559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a0142e00 con 0x7fb3a00b23e0 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb39d7fa640 1 -- 172.21.15.12:0/3104946559 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390004be0 con 0x7fb3a00b23e0 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 -- 172.21.15.12:0/3104946559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a01429e0 con 0x7fb3a00b23e0 2024-04-27T07:04:12.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb3a71c0640 1 -- 172.21.15.12:0/3104946559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a0069330 con 0x7fb3a00b23e0 2024-04-27T07:04:12.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb39d7fa640 1 -- 172.21.15.12:0/3104946559 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7fb390011070 con 0x7fb3a00b23e0 2024-04-27T07:04:12.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb39d7fa640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3800682f0 0x7fb38006a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:12.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.565+0000 7fb39d7fa640 1 -- 172.21.15.12:0/3104946559 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb390086070 con 0x7fb3a00b23e0 2024-04-27T07:04:12.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.569+0000 7fb3a4f5c640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3800682f0 0x7fb38006a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:12.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.569+0000 7fb3a4f5c640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3800682f0 0x7fb38006a7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3980af000 tx=0x7fb3980a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:12.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.573+0000 7fb39d7fa640 1 -- 172.21.15.12:0/3104946559 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb390056af0 con 0x7fb3a00b23e0 2024-04-27T07:04:12.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:12.733+0000 7fb3a71c0640 1 -- 172.21.15.12:0/3104946559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb3a0142fd0 con 0x7fb3a00b23e0 2024-04-27T07:04:13.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.561+0000 7fb39d7fa640 1 -- 172.21.15.12:0/3104946559 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fb3900563f0 con 0x7fb3a00b23e0 2024-04-27T07:04:13.567 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2024-04-27T07:04:13.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 -- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3800682f0 msgr2=0x7fb38006a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3800682f0 0x7fb38006a7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb3980af000 tx=0x7fb3980a7070 comp rx=0 tx=0).stop 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 -- 172.21.15.12:0/3104946559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a00b23e0 msgr2=0x7fb3a01422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a00b23e0 0x7fb3a01422d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb39000c3e0 tx=0x7fb39000c8a0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 -- 172.21.15.12:0/3104946559 shutdown_connections 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3800682f0 0x7fb38006a7b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3a00b23e0 0x7fb3a01422d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb3a0145640 0x7fb3a00b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 --2- 172.21.15.12:0/3104946559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3a0067b70 0x7fb3a00720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 -- 172.21.15.12:0/3104946559 >> 172.21.15.12:0/3104946559 conn(0x7fb3a006cac0 msgr2=0x7fb3a009ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 -- 172.21.15.12:0/3104946559 shutdown_connections 2024-04-27T07:04:13.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.565+0000 7fb3a71c0640 1 -- 172.21.15.12:0/3104946559 wait complete. 2024-04-27T07:04:13.599 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-27T07:04:13.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f694a98a640 1 Processor -- start 2024-04-27T07:04:13.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f694a98a640 1 -- start start 2024-04-27T07:04:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f694a98a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69440b21d0 0x7f69440b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f694a98a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f694406cc90 0x7f6944152600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f694a98a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69440b25b0 con 0x7f69440b53a0 2024-04-27T07:04:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f694a98a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f694406ece0 con 0x7f694406cc90 2024-04-27T07:04:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f694a98a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f694406ea00 con 0x7f69440b21d0 2024-04-27T07:04:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f6948f27640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f694406cc90 0x7f6944152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:13.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f693bfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f69440b53a0 0x7f69441483e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34792/0 (socket says 172.21.15.12:34792) 2024-04-27T07:04:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f693bfff640 1 -- 172.21.15.12:0/3253453375 learned_addr learned my addr 172.21.15.12:0/3253453375 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f6943fff640 1 --2- 172.21.15.12:0/3253453375 >> v2:172.21.15.53:3300/0 conn(0x7f69440b21d0 0x7f69440b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f6948f27640 1 -- 172.21.15.12:0/3253453375 >> v2:172.21.15.53:3300/0 conn(0x7f69440b21d0 msgr2=0x7f69440b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f6948f27640 1 --2- 172.21.15.12:0/3253453375 >> v2:172.21.15.53:3300/0 conn(0x7f69440b21d0 0x7f69440b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f6948f27640 1 -- 172.21.15.12:0/3253453375 >> v1:172.21.15.12:6789/0 conn(0x7f69440b53a0 legacy=0x7f69441483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f6948f27640 1 -- 172.21.15.12:0/3253453375 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6944147ec0 con 0x7f694406cc90 2024-04-27T07:04:13.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.733+0000 7f6943fff640 1 --2- 172.21.15.12:0/3253453375 >> v2:172.21.15.53:3300/0 conn(0x7f69440b21d0 0x7f69440b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f6948f27640 1 --2- 172.21.15.12:0/3253453375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f694406cc90 0x7f6944152600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f69440bc520 tx=0x7f693c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eced87a0092cb95 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f69437fe640 1 -- 172.21.15.12:0/3253453375 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c0b2070 con 0x7f694406cc90 2024-04-27T07:04:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f69437fe640 1 -- 172.21.15.12:0/3253453375 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f693c0a5760 con 0x7f694406cc90 2024-04-27T07:04:13.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f69437fe640 1 -- 172.21.15.12:0/3253453375 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c0a5b30 con 0x7f694406cc90 2024-04-27T07:04:13.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- 172.21.15.12:0/3253453375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f694406cc90 msgr2=0x7f6944152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:13.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 --2- 172.21.15.12:0/3253453375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f694406cc90 0x7f6944152600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f69440bc520 tx=0x7f693c01c8a0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- 172.21.15.12:0/3253453375 shutdown_connections 2024-04-27T07:04:13.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 --2- 172.21.15.12:0/3253453375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f694406cc90 0x7f6944152600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 --2- 172.21.15.12:0/3253453375 >> v2:172.21.15.53:3300/0 conn(0x7f69440b21d0 0x7f69440b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- 172.21.15.12:0/3253453375 >> 172.21.15.12:0/3253453375 conn(0x7f69440bb4f0 msgr2=0x7f69440bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:13.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- 172.21.15.12:0/3253453375 shutdown_connections 2024-04-27T07:04:13.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- 172.21.15.12:0/3253453375 wait complete. 2024-04-27T07:04:13.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 Processor -- start 2024-04-27T07:04:13.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- start start 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f694406cc90 0x7f6944140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69440b21d0 0x7f69441dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69440b53a0 0x7f694414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69441556f0 con 0x7f69440b21d0 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6944155570 con 0x7f69440b53a0 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f6944155870 con 0x7f694406cc90 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f693bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69440b21d0 0x7f69441dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f693bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69440b21d0 0x7f69441dffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38358/0 (socket says 172.21.15.12:38358) 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f693bfff640 1 -- 172.21.15.12:0/1376476839 learned_addr learned my addr 172.21.15.12:0/1376476839 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f6948f27640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69440b53a0 0x7f694414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:13.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f6943fff640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f694406cc90 0x7f6944140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f693bfff640 1 -- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f694406cc90 msgr2=0x7f6944140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f693bfff640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f694406cc90 0x7f6944140b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f693bfff640 1 -- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69440b53a0 msgr2=0x7f694414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f693bfff640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69440b53a0 0x7f694414b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f693bfff640 1 -- 172.21.15.12:0/1376476839 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69441499d0 con 0x7f69440b21d0 2024-04-27T07:04:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f6943fff640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f694406cc90 0x7f6944140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f6948f27640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69440b53a0 0x7f694414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f693bfff640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69440b21d0 0x7f69441dffa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f692c001ec0 tx=0x7f692c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f69417fa640 1 -- 172.21.15.12:0/1376476839 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c017070 con 0x7f69440b21d0 2024-04-27T07:04:13.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f69417fa640 1 -- 172.21.15.12:0/1376476839 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692c005cf0 con 0x7f69440b21d0 2024-04-27T07:04:13.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f69417fa640 1 -- 172.21.15.12:0/1376476839 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0072d0 con 0x7f69440b21d0 2024-04-27T07:04:13.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- 172.21.15.12:0/1376476839 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694413f9f0 con 0x7f69440b21d0 2024-04-27T07:04:13.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.737+0000 7f694a98a640 1 -- 172.21.15.12:0/1376476839 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f694414bf90 con 0x7f69440b21d0 2024-04-27T07:04:13.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.741+0000 7f694a98a640 1 -- 172.21.15.12:0/1376476839 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69441e04e0 con 0x7f69440b21d0 2024-04-27T07:04:13.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.741+0000 7f69417fa640 1 -- 172.21.15.12:0/1376476839 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f692c01a070 con 0x7f69440b21d0 2024-04-27T07:04:13.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.741+0000 7f69417fa640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6924068380 0x7f692406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:13.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.741+0000 7f69417fa640 1 -- 172.21.15.12:0/1376476839 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f692c085ac0 con 0x7f69440b21d0 2024-04-27T07:04:13.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.741+0000 7f6943fff640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6924068380 0x7f692406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:13.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.741+0000 7f6943fff640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6924068380 0x7f692406a840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6934008930 tx=0x7f6934008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:13.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.745+0000 7f69417fa640 1 -- 172.21.15.12:0/1376476839 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f692c055b80 con 0x7f69440b21d0 2024-04-27T07:04:13.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:13.905+0000 7f694a98a640 1 -- 172.21.15.12:0/1376476839 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6944069330 con 0x7f69440b21d0 2024-04-27T07:04:14.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.573+0000 7f69417fa640 1 -- 172.21.15.12:0/1376476839 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f692c059e80 con 0x7f69440b21d0 2024-04-27T07:04:14.578 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2024-04-27T07:04:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 -- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6924068380 msgr2=0x7f692406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6924068380 0x7f692406a840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6934008930 tx=0x7f6934008040 comp rx=0 tx=0).stop 2024-04-27T07:04:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 -- 172.21.15.12:0/1376476839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69440b21d0 msgr2=0x7f69441dffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69440b21d0 0x7f69441dffa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f692c001ec0 tx=0x7f692c00c890 comp rx=0 tx=0).stop 2024-04-27T07:04:14.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 -- 172.21.15.12:0/1376476839 shutdown_connections 2024-04-27T07:04:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6924068380 0x7f692406a840 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69440b53a0 0x7f694414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69440b21d0 0x7f69441dffa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 --2- 172.21.15.12:0/1376476839 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f694406cc90 0x7f6944140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 -- 172.21.15.12:0/1376476839 >> 172.21.15.12:0/1376476839 conn(0x7f69440bb4f0 msgr2=0x7f6944145c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 -- 172.21.15.12:0/1376476839 shutdown_connections 2024-04-27T07:04:14.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.577+0000 7f694a98a640 1 -- 172.21.15.12:0/1376476839 wait complete. 2024-04-27T07:04:14.616 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-27T07:04:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5859bc4640 1 Processor -- start 2024-04-27T07:04:14.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5859bc4640 1 -- start start 2024-04-27T07:04:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5859bc4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f58540b60a0 0x7f585414c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5859bc4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540b6480 0x7f5854156a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5859bc4640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f585409e4d0 con 0x7f5854150450 2024-04-27T07:04:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5859bc4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f585409c460 con 0x7f58540b60a0 2024-04-27T07:04:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5859bc4640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f585414f970 con 0x7f58540b6480 2024-04-27T07:04:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5852ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5854150450 0x7f585409ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34802/0 (socket says 172.21.15.12:34802) 2024-04-27T07:04:14.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5852ffd640 1 -- 172.21.15.12:0/4130926320 learned_addr learned my addr 172.21.15.12:0/4130926320 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5853fff640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540b6480 0x7f5854156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58537fe640 1 --2- 172.21.15.12:0/4130926320 >> v2:172.21.15.53:3300/0 conn(0x7f58540b60a0 0x7f585414c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1663124729 0 0) 0x7f585409e4d0 con 0x7f5854150450 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f583c003530 con 0x7f5854150450 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 21741956 0 0) 0x7f583c003530 con 0x7f5854150450 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540b6480 msgr2=0x7f5854156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540b6480 0x7f5854156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 >> v2:172.21.15.53:3300/0 conn(0x7f58540b60a0 msgr2=0x7f585414c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 --2- 172.21.15.12:0/4130926320 >> v2:172.21.15.53:3300/0 conn(0x7f58540b60a0 0x7f585414c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585414fc50 con 0x7f5854150450 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f5844001680 con 0x7f5854150450 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 >> v1:172.21.15.12:6789/0 conn(0x7f5854150450 legacy=0x7f585409ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540b6480 0x7f583c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c003f40 0x7f583c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:14.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f583c004d50 0x7f583c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f585409c460 con 0x7f583c003f40 2024-04-27T07:04:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f585414f970 con 0x7f583c004d50 2024-04-27T07:04:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f583c00ba20 con 0x7f58540b6480 2024-04-27T07:04:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5852ffd640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c003f40 0x7f583c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f58537fe640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540b6480 0x7f583c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5853fff640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f583c004d50 0x7f583c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5852ffd640 1 -- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540b6480 msgr2=0x7f583c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:14.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5852ffd640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540b6480 0x7f583c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5852ffd640 1 -- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f583c004d50 msgr2=0x7f583c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:14.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5852ffd640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f583c004d50 0x7f583c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.741+0000 7f5852ffd640 1 -- 172.21.15.12:0/4130926320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583c003530 con 0x7f583c003f40 2024-04-27T07:04:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f5852ffd640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c003f40 0x7f583c004810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f584400c210 tx=0x7f584401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9679387286ef4e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844007d80 con 0x7f583c003f40 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58440a9020 con 0x7f583c003f40 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f58527fc640 1 -- 172.21.15.12:0/4130926320 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844005ae0 con 0x7f583c003f40 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f5859bc4640 1 -- 172.21.15.12:0/4130926320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c003f40 msgr2=0x7f583c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f5859bc4640 1 --2- 172.21.15.12:0/4130926320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c003f40 0x7f583c004810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f584400c210 tx=0x7f584401cca0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f5853fff640 1 -- 172.21.15.12:0/4130926320 reap_dead start 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f5859bc4640 1 -- 172.21.15.12:0/4130926320 shutdown_connections 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f5859bc4640 1 -- 172.21.15.12:0/4130926320 >> 172.21.15.12:0/4130926320 conn(0x7f58540b93b0 msgr2=0x7f58540b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f5859bc4640 1 -- 172.21.15.12:0/4130926320 shutdown_connections 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f5859bc4640 1 -- 172.21.15.12:0/4130926320 wait complete. 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f5859bc4640 1 Processor -- start 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.745+0000 7f5859bc4640 1 -- start start 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5859bc4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f583c003f40 0x7f58540b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5859bc4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58541e5af0 0x7f585414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:14.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5859bc4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540afc10 0x7f58540729d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5859bc4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58541574b0 con 0x7f58541e5af0 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5859bc4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58540b6c70 con 0x7f583c003f40 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5859bc4640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f5854157630 con 0x7f58540afc10 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58541e5af0 0x7f585414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58541e5af0 0x7f585414d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38384/0 (socket says 172.21.15.12:38384) 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5852ffd640 1 -- 172.21.15.12:0/1499600027 learned_addr learned my addr 172.21.15.12:0/1499600027 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f58537fe640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f583c003f40 0x7f58540b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5853fff640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540afc10 0x7f58540729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5852ffd640 1 -- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540afc10 msgr2=0x7f58540729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5852ffd640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540afc10 0x7f58540729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5852ffd640 1 -- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f583c003f40 msgr2=0x7f58540b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5852ffd640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f583c003f40 0x7f58540b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5852ffd640 1 -- 172.21.15.12:0/1499600027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585414d890 con 0x7f58541e5af0 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5853fff640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540afc10 0x7f58540729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:14.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5852ffd640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58541e5af0 0x7f585414d350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f584400c210 tx=0x7f583c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:14.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f583bfff640 1 -- 172.21.15.12:0/1499600027 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58440a5a90 con 0x7f58541e5af0 2024-04-27T07:04:14.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f583bfff640 1 -- 172.21.15.12:0/1499600027 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58440a5c30 con 0x7f58541e5af0 2024-04-27T07:04:14.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5859bc4640 1 -- 172.21.15.12:0/1499600027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58541e5010 con 0x7f58541e5af0 2024-04-27T07:04:14.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f583bfff640 1 -- 172.21.15.12:0/1499600027 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58440047f0 con 0x7f58541e5af0 2024-04-27T07:04:14.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5859bc4640 1 -- 172.21.15.12:0/1499600027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585414ea80 con 0x7f58541e5af0 2024-04-27T07:04:14.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f5859bc4640 1 -- 172.21.15.12:0/1499600027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5854068c20 con 0x7f58541e5af0 2024-04-27T07:04:14.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f583bfff640 1 -- 172.21.15.12:0/1499600027 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5844004990 con 0x7f58541e5af0 2024-04-27T07:04:14.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f583bfff640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5840068380 0x7f584006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:14.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.749+0000 7f583bfff640 1 -- 172.21.15.12:0/1499600027 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f584411fd70 con 0x7f58541e5af0 2024-04-27T07:04:14.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.753+0000 7f58537fe640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5840068380 0x7f584006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:14.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.753+0000 7f58537fe640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5840068380 0x7f584006a840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5848009a30 tx=0x7f5848008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:14.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.757+0000 7f583bfff640 1 -- 172.21.15.12:0/1499600027 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f58440169b0 con 0x7f58541e5af0 2024-04-27T07:04:14.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:14.957+0000 7f5859bc4640 1 -- 172.21.15.12:0/1499600027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f58540ba0c0 con 0x7f58541e5af0 2024-04-27T07:04:15.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.597+0000 7f583bfff640 1 -- 172.21.15.12:0/1499600027 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f58440f0420 con 0x7f58541e5af0 2024-04-27T07:04:15.606 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 2 and data pool 3 2024-04-27T07:04:15.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 -- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5840068380 msgr2=0x7f584006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:15.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5840068380 0x7f584006a840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5848009a30 tx=0x7f5848008040 comp rx=0 tx=0).stop 2024-04-27T07:04:15.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 -- 172.21.15.12:0/1499600027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58541e5af0 msgr2=0x7f585414d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:15.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58541e5af0 0x7f585414d350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f584400c210 tx=0x7f583c005840 comp rx=0 tx=0).stop 2024-04-27T07:04:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 -- 172.21.15.12:0/1499600027 shutdown_connections 2024-04-27T07:04:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5840068380 0x7f584006a840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58540afc10 0x7f58540729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58541e5af0 0x7f585414d350 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 --2- 172.21.15.12:0/1499600027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f583c003f40 0x7f58540b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 -- 172.21.15.12:0/1499600027 >> 172.21.15.12:0/1499600027 conn(0x7f58540b93b0 msgr2=0x7f585406dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 -- 172.21.15.12:0/1499600027 shutdown_connections 2024-04-27T07:04:15.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.609+0000 7f5859bc4640 1 -- 172.21.15.12:0/1499600027 wait complete. 2024-04-27T07:04:15.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:15.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.757+0000 7f73d452b640 1 Processor -- start 2024-04-27T07:04:15.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.757+0000 7f73d452b640 1 -- start start 2024-04-27T07:04:15.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d452b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f73cc0b62a0 0x7f73cc0b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d452b640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73cc148a60 0x7f73cc141830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d452b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73cc147db0 con 0x7f73cc1485b0 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d452b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f73cc148090 con 0x7f73cc0b62a0 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d452b640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f73cc14be80 con 0x7f73cc148a60 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d22c7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f73cc0b62a0 0x7f73cc0b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d22c7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f73cc0b62a0 0x7f73cc0b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38386/0 (socket says 172.21.15.12:38386) 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d22c7640 1 -- 172.21.15.12:0/2607010444 learned_addr learned my addr 172.21.15.12:0/2607010444 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d2ac8640 1 --2- 172.21.15.12:0/2607010444 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73cc148a60 0x7f73cc141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d22c7640 1 -- 172.21.15.12:0/2607010444 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73cc148a60 msgr2=0x7f73cc141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d22c7640 1 --2- 172.21.15.12:0/2607010444 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73cc148a60 0x7f73cc141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d22c7640 1 -- 172.21.15.12:0/2607010444 >> v1:172.21.15.53:6789/0 conn(0x7f73cc1485b0 legacy=0x7f73cc14b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:15.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d22c7640 1 -- 172.21.15.12:0/2607010444 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc14c170 con 0x7f73cc0b62a0 2024-04-27T07:04:15.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d22c7640 1 --2- 172.21.15.12:0/2607010444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc0b62a0 0x7f73cc0b6680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009870 tx=0x7f73c801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8cdc431f6f9c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:15.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d12c5640 1 -- 172.21.15.12:0/2607010444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c80b0070 con 0x7f73cc0b62a0 2024-04-27T07:04:15.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d12c5640 1 -- 172.21.15.12:0/2607010444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c80ab600 con 0x7f73cc0b62a0 2024-04-27T07:04:15.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d12c5640 1 -- 172.21.15.12:0/2607010444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c80ab9d0 con 0x7f73cc0b62a0 2024-04-27T07:04:15.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d452b640 1 -- 172.21.15.12:0/2607010444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc0b62a0 msgr2=0x7f73cc0b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:15.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.761+0000 7f73d452b640 1 --2- 172.21.15.12:0/2607010444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc0b62a0 0x7f73cc0b6680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009870 tx=0x7f73c801c4d0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- 172.21.15.12:0/2607010444 shutdown_connections 2024-04-27T07:04:15.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 --2- 172.21.15.12:0/2607010444 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73cc148a60 0x7f73cc141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 --2- 172.21.15.12:0/2607010444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc0b62a0 0x7f73cc0b6680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- 172.21.15.12:0/2607010444 >> 172.21.15.12:0/2607010444 conn(0x7f73cc068190 msgr2=0x7f73cc0bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:15.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- 172.21.15.12:0/2607010444 shutdown_connections 2024-04-27T07:04:15.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- 172.21.15.12:0/2607010444 wait complete. 2024-04-27T07:04:15.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 Processor -- start 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- start start 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73cc0b62a0 0x7f73cc1e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc1485b0 0x7f73cc1e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc148a60 0x7f73cc135320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1551d0 con 0x7f73cc1485b0 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc155050 con 0x7f73cc148a60 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f73cc155350 con 0x7f73cc0b62a0 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d1ac6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc1485b0 0x7f73cc1e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d1ac6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc1485b0 0x7f73cc1e3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38390/0 (socket says 172.21.15.12:38390) 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d1ac6640 1 -- 172.21.15.12:0/462137144 learned_addr learned my addr 172.21.15.12:0/462137144 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d22c7640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73cc0b62a0 0x7f73cc1e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d2ac8640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc148a60 0x7f73cc135320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d1ac6640 1 -- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73cc0b62a0 msgr2=0x7f73cc1e48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:15.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d1ac6640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73cc0b62a0 0x7f73cc1e48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d1ac6640 1 -- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc148a60 msgr2=0x7f73cc135320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d1ac6640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc148a60 0x7f73cc135320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d1ac6640 1 -- 172.21.15.12:0/462137144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc13a750 con 0x7f73cc1485b0 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d2ac8640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc148a60 0x7f73cc135320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d1ac6640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc1485b0 0x7f73cc1e3da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00c500 tx=0x7f73bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73c2ffd640 1 -- 172.21.15.12:0/462137144 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc016020 con 0x7f73cc1485b0 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73c2ffd640 1 -- 172.21.15.12:0/462137144 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73bc005220 con 0x7f73cc1485b0 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- 172.21.15.12:0/462137144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc1392e0 con 0x7f73cc1485b0 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73c2ffd640 1 -- 172.21.15.12:0/462137144 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc007030 con 0x7f73cc1485b0 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- 172.21.15.12:0/462137144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc1492b0 con 0x7f73cc1485b0 2024-04-27T07:04:15.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.765+0000 7f73d452b640 1 -- 172.21.15.12:0/462137144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73cc069330 con 0x7f73cc1485b0 2024-04-27T07:04:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.769+0000 7f73c2ffd640 1 -- 172.21.15.12:0/462137144 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f73bc0071f0 con 0x7f73cc1485b0 2024-04-27T07:04:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.769+0000 7f73c2ffd640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f73b00683b0 0x7f73b006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.769+0000 7f73c2ffd640 1 -- 172.21.15.12:0/462137144 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f73bc086380 con 0x7f73cc1485b0 2024-04-27T07:04:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.769+0000 7f73d22c7640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f73b00683b0 0x7f73b006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.769+0000 7f73d22c7640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f73b00683b0 0x7f73b006a870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73c80ad000 tx=0x7f73c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:15.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.773+0000 7f73c2ffd640 1 -- 172.21.15.12:0/462137144 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73bc020020 con 0x7f73cc1485b0 2024-04-27T07:04:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.937+0000 7f73d452b640 1 -- 172.21.15.12:0/462137144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73cc1478e0 con 0x7f73cc1485b0 2024-04-27T07:04:15.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.941+0000 7f73c2ffd640 1 -- 172.21.15.12:0/462137144 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16558 (secure 0 0 0) 0x7f73bc056950 con 0x7f73cc1485b0 2024-04-27T07:04:15.946 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:15.946 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:14.961559+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:15.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 -- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f73b00683b0 msgr2=0x7f73b006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f73b00683b0 0x7f73b006a870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73c80ad000 tx=0x7f73c8008040 comp rx=0 tx=0).stop 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 -- 172.21.15.12:0/462137144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc1485b0 msgr2=0x7f73cc1e3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc1485b0 0x7f73cc1e3da0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00c500 tx=0x7f73bc00c9c0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 -- 172.21.15.12:0/462137144 shutdown_connections 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f73b00683b0 0x7f73b006a870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc148a60 0x7f73cc135320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73cc1485b0 0x7f73cc1e3da0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 --2- 172.21.15.12:0/462137144 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73cc0b62a0 0x7f73cc1e48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 -- 172.21.15.12:0/462137144 >> 172.21.15.12:0/462137144 conn(0x7f73cc068190 msgr2=0x7f73cc0b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 -- 172.21.15.12:0/462137144 shutdown_connections 2024-04-27T07:04:15.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:15.945+0000 7f73d452b640 1 -- 172.21.15.12:0/462137144 wait complete. 2024-04-27T07:04:15.965 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.077+0000 7fcafb253640 1 Processor -- start 2024-04-27T07:04:16.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.077+0000 7fcafb253640 1 -- start start 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcafb253640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcaf40b3680 0x7fcaf4145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcafb253640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcaf41460b0 0x7fcaf4150410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcafb253640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf406dac0 con 0x7fcaf40b9a10 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcafb253640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf406e890 con 0x7fcaf41460b0 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcafb253640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf40b6cd0 con 0x7fcaf40b3680 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf8fef640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcaf40b9a10 0x7fcaf40b65c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34804/0 (socket says 172.21.15.12:34804) 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf8fef640 1 -- 172.21.15.12:0/3682064613 learned_addr learned my addr 172.21.15.12:0/3682064613 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf3fff640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcaf40b3680 0x7fcaf4145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf97f0640 1 --2- 172.21.15.12:0/3682064613 >> v2:172.21.15.53:3300/0 conn(0x7fcaf41460b0 0x7fcaf4150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540177781 0 0) 0x7fcadc001120 con 0x7fcaf40b9a10 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcae0003530 con 0x7fcaf40b9a10 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 504342637 0 0) 0x7fcae0003530 con 0x7fcaf40b9a10 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcaf40b3680 msgr2=0x7fcaf4145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:16.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcaf40b3680 0x7fcaf4145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 >> v2:172.21.15.53:3300/0 conn(0x7fcaf41460b0 msgr2=0x7fcaf4150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 --2- 172.21.15.12:0/3682064613 >> v2:172.21.15.53:3300/0 conn(0x7fcaf41460b0 0x7fcaf4150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf40b9df0 con 0x7fcaf40b9a10 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fcadc001870 con 0x7fcaf40b9a10 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 >> v1:172.21.15.12:6789/0 conn(0x7fcaf40b9a10 legacy=0x7fcaf40b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcaf40b3680 0x7fcae00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0003f40 0x7fcae0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcae0004d50 0x7fcae00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf406e890 con 0x7fcae0004d50 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40b6cd0 con 0x7fcae0003f40 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fcae000ba20 con 0x7fcaf40b3680 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf97f0640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcae0004d50 0x7fcae00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf8fef640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcaf40b3680 0x7fcae00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf3fff640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0003f40 0x7fcae0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf97f0640 1 -- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcaf40b3680 msgr2=0x7fcae00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:16.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf97f0640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcaf40b3680 0x7fcae00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf97f0640 1 -- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0003f40 msgr2=0x7fcae0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:16.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf97f0640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0003f40 0x7fcae0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf97f0640 1 -- 172.21.15.12:0/3682064613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae0003530 con 0x7fcae0004d50 2024-04-27T07:04:16.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.081+0000 7fcaf3fff640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae0003f40 0x7fcae0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcae0004d50 0x7fcae00051d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009b10 tx=0x7fcaec01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74281659970801c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0b2070 con 0x7fcae0004d50 2024-04-27T07:04:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec0a56a0 con 0x7fcae0004d50 2024-04-27T07:04:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf37fe640 1 -- 172.21.15.12:0/3682064613 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0096b0 con 0x7fcae0004d50 2024-04-27T07:04:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- 172.21.15.12:0/3682064613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcae0004d50 msgr2=0x7fcae00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 --2- 172.21.15.12:0/3682064613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcae0004d50 0x7fcae00051d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009b10 tx=0x7fcaec01cab0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 -- 172.21.15.12:0/3682064613 reap_dead start 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- 172.21.15.12:0/3682064613 shutdown_connections 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- 172.21.15.12:0/3682064613 >> 172.21.15.12:0/3682064613 conn(0x7fcaf4067d10 msgr2=0x7fcaf4068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- 172.21.15.12:0/3682064613 shutdown_connections 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- 172.21.15.12:0/3682064613 wait complete. 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 Processor -- start 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- start start 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae0004d50 0x7fcaf41434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4142a70 0x7fcaf413e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf413ee50 0x7fcaf4070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf406d300 con 0x7fcaf413ee50 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf406e370 con 0x7fcaf4142a70 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf406d480 con 0x7fcae0004d50 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf413ee50 0x7fcaf4070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf413ee50 0x7fcaf4070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38398/0 (socket says 172.21.15.12:38398) 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 -- 172.21.15.12:0/1665552913 learned_addr learned my addr 172.21.15.12:0/1665552913 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf8fef640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae0004d50 0x7fcaf41434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf3fff640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4142a70 0x7fcaf413e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 -- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae0004d50 msgr2=0x7fcaf41434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae0004d50 0x7fcaf41434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 -- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4142a70 msgr2=0x7fcaf413e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4142a70 0x7fcaf413e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 -- 172.21.15.12:0/1665552913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf4145820 con 0x7fcaf413ee50 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf97f0640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf413ee50 0x7fcaf4070440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0b0040 tx=0x7fcae0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf17fa640 1 -- 172.21.15.12:0/1665552913 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0b2040 con 0x7fcaf413ee50 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf17fa640 1 -- 172.21.15.12:0/1665552913 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec004020 con 0x7fcaf413ee50 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- 172.21.15.12:0/1665552913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf4145540 con 0x7fcaf413ee50 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcaf17fa640 1 -- 172.21.15.12:0/1665552913 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0096b0 con 0x7fcaf413ee50 2024-04-27T07:04:16.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.085+0000 7fcafb253640 1 -- 172.21.15.12:0/1665552913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf40716b0 con 0x7fcaf413ee50 2024-04-27T07:04:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.089+0000 7fcafb253640 1 -- 172.21.15.12:0/1665552913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf4068c20 con 0x7fcaf413ee50 2024-04-27T07:04:16.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.089+0000 7fcaf17fa640 1 -- 172.21.15.12:0/1665552913 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcaec0bf060 con 0x7fcaf413ee50 2024-04-27T07:04:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.089+0000 7fcaf17fa640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 0x7fcad406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.089+0000 7fcaf17fa640 1 -- 172.21.15.12:0/1665552913 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fcaec120500 con 0x7fcaf413ee50 2024-04-27T07:04:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.089+0000 7fcaf8fef640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 0x7fcad406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.089+0000 7fcaf8fef640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 0x7fcad406a840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcadc001060 tx=0x7fcadc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:16.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.093+0000 7fcaf17fa640 1 -- 172.21.15.12:0/1665552913 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcaec016800 con 0x7fcaf413ee50 2024-04-27T07:04:16.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.257+0000 7fcafb253640 1 -- 172.21.15.12:0/1665552913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcaf406d7f0 con 0x7fcaf413ee50 2024-04-27T07:04:16.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcaf17fa640 1 -- 172.21.15.12:0/1665552913 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16558 (secure 0 0 0) 0x7fcaec0f0ab0 con 0x7fcaf413ee50 2024-04-27T07:04:16.265 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:16.265 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:14.961559+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 -- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 msgr2=0x7fcad406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 0x7fcad406a840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcadc001060 tx=0x7fcadc009040 comp rx=0 tx=0).stop 2024-04-27T07:04:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 -- 172.21.15.12:0/1665552913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf413ee50 msgr2=0x7fcaf4070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf413ee50 0x7fcaf4070440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0b0040 tx=0x7fcae0005840 comp rx=0 tx=0).stop 2024-04-27T07:04:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 -- 172.21.15.12:0/1665552913 shutdown_connections 2024-04-27T07:04:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 0x7fcad406a840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf413ee50 0x7fcaf4070440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcaf4142a70 0x7fcaf413e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 --2- 172.21.15.12:0/1665552913 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae0004d50 0x7fcaf41434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 -- 172.21.15.12:0/1665552913 >> 172.21.15.12:0/1665552913 conn(0x7fcaf4067d10 msgr2=0x7fcaf40af170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:16.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 -- 172.21.15.12:0/1665552913 shutdown_connections 2024-04-27T07:04:16.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.261+0000 7fcafb253640 1 -- 172.21.15.12:0/1665552913 wait complete. 2024-04-27T07:04:16.284 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-27T07:04:16.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0fade6640 1 Processor -- start 2024-04-27T07:04:16.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0fade6640 1 -- start start 2024-04-27T07:04:16.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0fade6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0f406e720 0x7fb0f40b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0fade6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0f40b7b90 con 0x7fb0f414ba10 2024-04-27T07:04:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0fade6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0f40b5330 con 0x7fb0f40b5810 2024-04-27T07:04:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0fade6640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f406eb00 con 0x7fb0f406e720 2024-04-27T07:04:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f9383640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb0f414ba10 0x7fb0f41528b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34810/0 (socket says 172.21.15.12:34810) 2024-04-27T07:04:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f9383640 1 -- 172.21.15.12:0/3432178806 learned_addr learned my addr 172.21.15.12:0/3432178806 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f8b82640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0f406e720 0x7fb0f40b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459959249 0 0) 0x7fb0f40b7b90 con 0x7fb0f414ba10 2024-04-27T07:04:16.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0dc003530 con 0x7fb0f414ba10 2024-04-27T07:04:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 101454492 0 0) 0x7fb0dc003530 con 0x7fb0f414ba10 2024-04-27T07:04:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0f406e720 msgr2=0x7fb0f40b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0f406e720 0x7fb0f40b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 >> v1:172.21.15.53:6789/0 conn(0x7fb0f40b5810 legacy=0x7fb0f414b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f40b6870 con 0x7fb0f414ba10 2024-04-27T07:04:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fb0ec001a00 con 0x7fb0f414ba10 2024-04-27T07:04:16.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 >> v1:172.21.15.12:6789/0 conn(0x7fb0f414ba10 legacy=0x7fb0f41528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0dc0039f0 0x7fb0dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0dc0043a0 0x7fb0dc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0f406e720 0x7fb0dc005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f406eb00 con 0x7fb0dc0039f0 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc00ba80 con 0x7fb0dc0043a0 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fb0dc00bc80 con 0x7fb0f406e720 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f8b82640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0dc0039f0 0x7fb0dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f9383640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0f406e720 0x7fb0dc005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f3fff640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0dc0043a0 0x7fb0dc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f8b82640 1 -- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0f406e720 msgr2=0x7fb0dc005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f8b82640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0f406e720 0x7fb0dc005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f8b82640 1 -- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0dc0043a0 msgr2=0x7fb0dc004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f8b82640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0dc0043a0 0x7fb0dc004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f8b82640 1 -- 172.21.15.12:0/3432178806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc003530 con 0x7fb0dc0039f0 2024-04-27T07:04:16.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f9383640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0f406e720 0x7fb0dc005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:16.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.413+0000 7fb0f3fff640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0dc0043a0 0x7fb0dc004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0f8b82640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0dc0039f0 0x7fb0dc003dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009ae0 tx=0x7fb0e401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05fef80934cbec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e40096b0 con 0x7fb0dc0039f0 2024-04-27T07:04:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e40a6450 con 0x7fb0dc0039f0 2024-04-27T07:04:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0f37fe640 1 -- 172.21.15.12:0/3432178806 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e40a6840 con 0x7fb0dc0039f0 2024-04-27T07:04:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 -- 172.21.15.12:0/3432178806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0dc0039f0 msgr2=0x7fb0dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 --2- 172.21.15.12:0/3432178806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0dc0039f0 0x7fb0dc003dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009ae0 tx=0x7fb0e401ca70 comp rx=0 tx=0).stop 2024-04-27T07:04:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0f9383640 1 -- 172.21.15.12:0/3432178806 reap_dead start 2024-04-27T07:04:16.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 -- 172.21.15.12:0/3432178806 shutdown_connections 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 -- 172.21.15.12:0/3432178806 >> 172.21.15.12:0/3432178806 conn(0x7fb0f40bb560 msgr2=0x7fb0f40bb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 -- 172.21.15.12:0/3432178806 shutdown_connections 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 -- 172.21.15.12:0/3432178806 wait complete. 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 Processor -- start 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 -- start start 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0dc0039f0 0x7fb0f414a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f41e40a0 0x7fb0f413d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0f41e4520 0x7fb0f413d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f406cc90 con 0x7fb0f41e40a0 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40b6fa0 con 0x7fb0f41e4520 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0fade6640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f41547e0 con 0x7fb0dc0039f0 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0f3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f41e40a0 0x7fb0f413d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0f3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f41e40a0 0x7fb0f413d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38410/0 (socket says 172.21.15.12:38410) 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0f3fff640 1 -- 172.21.15.12:0/2586675439 learned_addr learned my addr 172.21.15.12:0/2586675439 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:16.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0f8b82640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0dc0039f0 0x7fb0f414a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.417+0000 7fb0f9383640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0f41e4520 0x7fb0f413d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f3fff640 1 -- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0dc0039f0 msgr2=0x7fb0f414a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f3fff640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0dc0039f0 0x7fb0f414a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f3fff640 1 -- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0f41e4520 msgr2=0x7fb0f413d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f3fff640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0f41e4520 0x7fb0f413d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f3fff640 1 -- 172.21.15.12:0/2586675439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f414eeb0 con 0x7fb0f41e40a0 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f9383640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0f41e4520 0x7fb0f413d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f3fff640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f41e40a0 0x7fb0f413d2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800d740 tx=0x7fb0e800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f17fa640 1 -- 172.21.15.12:0/2586675439 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8005480 con 0x7fb0f41e40a0 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f17fa640 1 -- 172.21.15.12:0/2586675439 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e8017070 con 0x7fb0f41e40a0 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f17fa640 1 -- 172.21.15.12:0/2586675439 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800b850 con 0x7fb0f41e40a0 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0fade6640 1 -- 172.21.15.12:0/2586675439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f4150040 con 0x7fb0f41e40a0 2024-04-27T07:04:16.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0fade6640 1 -- 172.21.15.12:0/2586675439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f4145450 con 0x7fb0f41e40a0 2024-04-27T07:04:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0fade6640 1 -- 172.21.15.12:0/2586675439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f4069330 con 0x7fb0f41e40a0 2024-04-27T07:04:16.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f17fa640 1 -- 172.21.15.12:0/2586675439 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb0e800b9f0 con 0x7fb0f41e40a0 2024-04-27T07:04:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f17fa640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb0e0068380 0x7fb0e006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f17fa640 1 -- 172.21.15.12:0/2586675439 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fb0e8086900 con 0x7fb0f41e40a0 2024-04-27T07:04:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f8b82640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb0e0068380 0x7fb0e006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:16.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.421+0000 7fb0f8b82640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb0e0068380 0x7fb0e006a840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40ad000 tx=0x7fb0e40b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:16.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.425+0000 7fb0f17fa640 1 -- 172.21.15.12:0/2586675439 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb0e8056f50 con 0x7fb0f41e40a0 2024-04-27T07:04:16.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:16.637+0000 7fb0fade6640 1 -- 172.21.15.12:0/2586675439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb0f40bc080 con 0x7fb0f41e40a0 2024-04-27T07:04:17.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.605+0000 7fb0f17fa640 1 -- 172.21.15.12:0/2586675439 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fb0e8056850 con 0x7fb0f41e40a0 2024-04-27T07:04:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 -- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb0e0068380 msgr2=0x7fb0e006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb0e0068380 0x7fb0e006a840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40ad000 tx=0x7fb0e40b2040 comp rx=0 tx=0).stop 2024-04-27T07:04:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 -- 172.21.15.12:0/2586675439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f41e40a0 msgr2=0x7fb0f413d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f41e40a0 0x7fb0f413d2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800d740 tx=0x7fb0e800dc00 comp rx=0 tx=0).stop 2024-04-27T07:04:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 -- 172.21.15.12:0/2586675439 shutdown_connections 2024-04-27T07:04:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb0e0068380 0x7fb0e006a840 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0f41e4520 0x7fb0f413d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:17.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f41e40a0 0x7fb0f413d2e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:17.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 --2- 172.21.15.12:0/2586675439 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb0dc0039f0 0x7fb0f414a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:17.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 -- 172.21.15.12:0/2586675439 >> 172.21.15.12:0/2586675439 conn(0x7fb0f40bb560 msgr2=0x7fb0f40b87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:17.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 -- 172.21.15.12:0/2586675439 shutdown_connections 2024-04-27T07:04:17.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.609+0000 7fb0fade6640 1 -- 172.21.15.12:0/2586675439 wait complete. 2024-04-27T07:04:17.635 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-27T07:04:17.635 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-27T07:04:17.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41ce6d8640 1 Processor -- start 2024-04-27T07:04:17.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41ce6d8640 1 -- start start 2024-04-27T07:04:17.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41ce6d8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c806e790 0x7f41c80b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41ce6d8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f41c80b5670 0x7f41c814a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41ce6d8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c806d000 con 0x7f41c814acd0 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41ce6d8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f41c80b2740 con 0x7f41c80b5670 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41ce6d8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f41c80b7d20 con 0x7f41c806e790 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41c77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f41c80b5670 0x7f41c814a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41c77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f41c80b5670 0x7f41c814a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41846/0 (socket says 172.21.15.12:41846) 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41c77fe640 1 -- 172.21.15.12:0/47040403 learned_addr learned my addr 172.21.15.12:0/47040403 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41c7fff640 1 --2- 172.21.15.12:0/47040403 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c806e790 0x7f41c80b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41c77fe640 1 -- 172.21.15.12:0/47040403 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c806e790 msgr2=0x7f41c80b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41c77fe640 1 --2- 172.21.15.12:0/47040403 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c806e790 0x7f41c80b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41c77fe640 1 -- 172.21.15.12:0/47040403 >> v1:172.21.15.53:6789/0 conn(0x7f41c814acd0 legacy=0x7f41c814b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41c77fe640 1 -- 172.21.15.12:0/47040403 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c814b140 con 0x7f41c80b5670 2024-04-27T07:04:17.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.765+0000 7f41c7fff640 1 --2- 172.21.15.12:0/47040403 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c806e790 0x7f41c80b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41c77fe640 1 --2- 172.21.15.12:0/47040403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c80b5670 0x7f41c814a790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009870 tx=0x7f41b801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9712089be69414 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41c6ffd640 1 -- 172.21.15.12:0/47040403 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80b0070 con 0x7f41c80b5670 2024-04-27T07:04:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41c6ffd640 1 -- 172.21.15.12:0/47040403 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b80ab600 con 0x7f41c80b5670 2024-04-27T07:04:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41c6ffd640 1 -- 172.21.15.12:0/47040403 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80ab9d0 con 0x7f41c80b5670 2024-04-27T07:04:17.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 -- 172.21.15.12:0/47040403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c80b5670 msgr2=0x7f41c814a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:17.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 --2- 172.21.15.12:0/47040403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c80b5670 0x7f41c814a790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009870 tx=0x7f41b801c440 comp rx=0 tx=0).stop 2024-04-27T07:04:17.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 -- 172.21.15.12:0/47040403 shutdown_connections 2024-04-27T07:04:17.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 --2- 172.21.15.12:0/47040403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c80b5670 0x7f41c814a790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:17.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 --2- 172.21.15.12:0/47040403 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c806e790 0x7f41c80b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:17.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 -- 172.21.15.12:0/47040403 >> 172.21.15.12:0/47040403 conn(0x7f41c8068210 msgr2=0x7f41c80bb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:17.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 -- 172.21.15.12:0/47040403 shutdown_connections 2024-04-27T07:04:17.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 -- 172.21.15.12:0/47040403 wait complete. 2024-04-27T07:04:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 Processor -- start 2024-04-27T07:04:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 -- start start 2024-04-27T07:04:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c806e790 0x7f41c813df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c80b5670 0x7f41c813b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41c814acd0 0x7f41c813b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41c806d670 con 0x7f41c806e790 2024-04-27T07:04:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f41c80b2460 con 0x7f41c814acd0 2024-04-27T07:04:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.769+0000 7f41ce6d8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f41c806d9d0 con 0x7f41c80b5670 2024-04-27T07:04:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c806e790 0x7f41c813df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c806e790 0x7f41c813df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41858/0 (socket says 172.21.15.12:41858) 2024-04-27T07:04:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c7fff640 1 -- 172.21.15.12:0/2237957181 learned_addr learned my addr 172.21.15.12:0/2237957181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:17.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41ccc75640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41c814acd0 0x7f41c813b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c77fe640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c80b5670 0x7f41c813b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c7fff640 1 -- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c80b5670 msgr2=0x7f41c813b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c7fff640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c80b5670 0x7f41c813b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c7fff640 1 -- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41c814acd0 msgr2=0x7f41c813b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c7fff640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41c814acd0 0x7f41c813b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c7fff640 1 -- 172.21.15.12:0/2237957181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c813f270 con 0x7f41c806e790 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c7fff640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c806e790 0x7f41c813df60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e810 tx=0x7f41bc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c4ff9640 1 -- 172.21.15.12:0/2237957181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc017070 con 0x7f41c806e790 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c4ff9640 1 -- 172.21.15.12:0/2237957181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41bc00c8d0 con 0x7f41c806e790 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c4ff9640 1 -- 172.21.15.12:0/2237957181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00f040 con 0x7f41c806e790 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41ce6d8640 1 -- 172.21.15.12:0/2237957181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c81e1050 con 0x7f41c806e790 2024-04-27T07:04:17.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41ce6d8640 1 -- 172.21.15.12:0/2237957181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c813f550 con 0x7f41c806e790 2024-04-27T07:04:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41ce6d8640 1 -- 172.21.15.12:0/2237957181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c8069330 con 0x7f41c806e790 2024-04-27T07:04:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c4ff9640 1 -- 172.21.15.12:0/2237957181 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f41bc01a070 con 0x7f41c806e790 2024-04-27T07:04:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c4ff9640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f41b4068380 0x7f41b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c4ff9640 1 -- 172.21.15.12:0/2237957181 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f41bc086620 con 0x7f41c806e790 2024-04-27T07:04:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.773+0000 7f41c77fe640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f41b4068380 0x7f41b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:17.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.777+0000 7f41c77fe640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f41b4068380 0x7f41b406a840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41b80ad000 tx=0x7f41b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:17.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.777+0000 7f41c4ff9640 1 -- 172.21.15.12:0/2237957181 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f41bc056bd0 con 0x7f41c806e790 2024-04-27T07:04:17.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:17.977+0000 7f41ce6d8640 1 -- 172.21.15.12:0/2237957181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f41c80ba330 con 0x7f41c806e790 2024-04-27T07:04:18.540 INFO:tasks.ceph.mds.a.smithi012.stderr:2024-04-27T07:04:18.537+0000 7fd8e37d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-27T07:04:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.621+0000 7f41c4ff9640 1 -- 172.21.15.12:0/2237957181 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f41bc0564d0 con 0x7f41c806e790 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 -- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f41b4068380 msgr2=0x7f41b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f41b4068380 0x7f41b406a840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41b80ad000 tx=0x7f41b8008040 comp rx=0 tx=0).stop 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 -- 172.21.15.12:0/2237957181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c806e790 msgr2=0x7f41c813df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c806e790 0x7f41c813df60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e810 tx=0x7f41bc00ecd0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 -- 172.21.15.12:0/2237957181 shutdown_connections 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f41b4068380 0x7f41b406a840 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41c814acd0 0x7f41c813b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f41c80b5670 0x7f41c813b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 --2- 172.21.15.12:0/2237957181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41c806e790 0x7f41c813df60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 -- 172.21.15.12:0/2237957181 >> 172.21.15.12:0/2237957181 conn(0x7f41c8068210 msgr2=0x7f41c80bab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 -- 172.21.15.12:0/2237957181 shutdown_connections 2024-04-27T07:04:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.625+0000 7f41ce6d8640 1 -- 172.21.15.12:0/2237957181 wait complete. 2024-04-27T07:04:18.651 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-27T07:04:18.652 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:04:18.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567c481640 1 Processor -- start 2024-04-27T07:04:18.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567c481640 1 -- start start 2024-04-27T07:04:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567c481640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56740b5ae0 0x7f56740b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567c481640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5674148810 0x7f5674152640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567c481640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56740b7560 con 0x7f56740b64c0 2024-04-27T07:04:18.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567c481640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f5674148c80 con 0x7f5674148810 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567c481640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f56740b77d0 con 0x7f56740b5ae0 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567aa1e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5674148810 0x7f5674152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567a21d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f56740b64c0 0x7f5674141ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40786/0 (socket says 172.21.15.12:40786) 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567a21d640 1 -- 172.21.15.12:0/2956540233 learned_addr learned my addr 172.21.15.12:0/2956540233 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f5679a1c640 1 --2- 172.21.15.12:0/2956540233 >> v2:172.21.15.53:3300/0 conn(0x7f56740b5ae0 0x7f56740b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567921b640 1 -- 172.21.15.12:0/2956540233 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 647706999 0 0) 0x7f56740b7560 con 0x7f56740b64c0 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567921b640 1 -- 172.21.15.12:0/2956540233 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5660003530 con 0x7f56740b64c0 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567aa1e640 1 -- 172.21.15.12:0/2956540233 >> v2:172.21.15.53:3300/0 conn(0x7f56740b5ae0 msgr2=0x7f56740b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567aa1e640 1 --2- 172.21.15.12:0/2956540233 >> v2:172.21.15.53:3300/0 conn(0x7f56740b5ae0 0x7f56740b7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567aa1e640 1 -- 172.21.15.12:0/2956540233 >> v1:172.21.15.12:6789/0 conn(0x7f56740b64c0 legacy=0x7f5674141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567aa1e640 1 -- 172.21.15.12:0/2956540233 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56740b68a0 con 0x7f5674148810 2024-04-27T07:04:18.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f5679a1c640 1 --2- 172.21.15.12:0/2956540233 >> v2:172.21.15.53:3300/0 conn(0x7f56740b5ae0 0x7f56740b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.777+0000 7f567aa1e640 1 --2- 172.21.15.12:0/2956540233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5674148810 0x7f5674152640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f56740b48b0 tx=0x7f566c01c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f061c543843858b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567921b640 1 -- 172.21.15.12:0/2956540233 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c0b2070 con 0x7f5674148810 2024-04-27T07:04:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567921b640 1 -- 172.21.15.12:0/2956540233 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f566c0a58a0 con 0x7f5674148810 2024-04-27T07:04:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567921b640 1 -- 172.21.15.12:0/2956540233 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c0a5c50 con 0x7f5674148810 2024-04-27T07:04:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- 172.21.15.12:0/2956540233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5674148810 msgr2=0x7f5674152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 --2- 172.21.15.12:0/2956540233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5674148810 0x7f5674152640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f56740b48b0 tx=0x7f566c01c7e0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- 172.21.15.12:0/2956540233 shutdown_connections 2024-04-27T07:04:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 --2- 172.21.15.12:0/2956540233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5674148810 0x7f5674152640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 --2- 172.21.15.12:0/2956540233 >> v2:172.21.15.53:3300/0 conn(0x7f56740b5ae0 0x7f56740b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- 172.21.15.12:0/2956540233 >> 172.21.15.12:0/2956540233 conn(0x7f5674073c00 msgr2=0x7f56740bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- 172.21.15.12:0/2956540233 shutdown_connections 2024-04-27T07:04:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- 172.21.15.12:0/2956540233 wait complete. 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 Processor -- start 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- start start 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56740b5ae0 0x7f567413eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56740b64c0 0x7f56741e0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5674148810 0x7f56741e0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56740b51c0 con 0x7f56740b5ae0 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f56740b5040 con 0x7f56740b64c0 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f56741421e0 con 0x7f5674148810 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567a21d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56740b5ae0 0x7f567413eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567a21d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56740b5ae0 0x7f567413eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41880/0 (socket says 172.21.15.12:41880) 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567a21d640 1 -- 172.21.15.12:0/57735238 learned_addr learned my addr 172.21.15.12:0/57735238 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567a21d640 1 -- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5674148810 msgr2=0x7f56741e0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567a21d640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5674148810 0x7f56741e0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567a21d640 1 -- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56740b64c0 msgr2=0x7f56741e0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567a21d640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56740b64c0 0x7f56741e0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567a21d640 1 -- 172.21.15.12:0/57735238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5674149b80 con 0x7f56740b5ae0 2024-04-27T07:04:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567a21d640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56740b5ae0 0x7f567413eaf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56700010f0 tx=0x7f567000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f566affd640 1 -- 172.21.15.12:0/57735238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670017070 con 0x7f56740b5ae0 2024-04-27T07:04:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f566affd640 1 -- 172.21.15.12:0/57735238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5670005d10 con 0x7f56740b5ae0 2024-04-27T07:04:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- 172.21.15.12:0/57735238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5674144fb0 con 0x7f56740b5ae0 2024-04-27T07:04:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f566affd640 1 -- 172.21.15.12:0/57735238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670007310 con 0x7f56740b5ae0 2024-04-27T07:04:18.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.781+0000 7f567c481640 1 -- 172.21.15.12:0/57735238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5674149df0 con 0x7f56740b5ae0 2024-04-27T07:04:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.785+0000 7f567c481640 1 -- 172.21.15.12:0/57735238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56741e02e0 con 0x7f56740b5ae0 2024-04-27T07:04:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.785+0000 7f566affd640 1 -- 172.21.15.12:0/57735238 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f567001a070 con 0x7f56740b5ae0 2024-04-27T07:04:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.785+0000 7f566affd640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5658068380 0x7f565806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.785+0000 7f566affd640 1 -- 172.21.15.12:0/57735238 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5670085cf0 con 0x7f56740b5ae0 2024-04-27T07:04:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.785+0000 7f5679a1c640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5658068380 0x7f565806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.785+0000 7f5679a1c640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5658068380 0x7f565806a840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5664006e90 tx=0x7f5664008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:18.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.789+0000 7f566affd640 1 -- 172.21.15.12:0/57735238 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f567008a070 con 0x7f56740b5ae0 2024-04-27T07:04:18.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.985+0000 7f567c481640 1 -- 172.21.15.12:0/57735238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5674068c20 con 0x7f56740b5ae0 2024-04-27T07:04:18.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.985+0000 7f566affd640 1 -- 172.21.15.12:0/57735238 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3527 (secure 0 0 0) 0x7f5670056340 con 0x7f56740b5ae0 2024-04-27T07:04:18.992 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:18.992 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1908224930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1908224930},{"type":"v1","addr":"172.21.15.53:6835","nonce":1908224930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:04:14.961460+0000","modified":"2024-04-27T07:04:17.982838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4242":{"gid":4242,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4173401303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4173401303},{"type":"v1","addr":"172.21.15.12:6833","nonce":4173401303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4242,"qdb_cluster":[4242]},"id":1}]} 2024-04-27T07:04:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-04-27T07:04:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 -- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5658068380 msgr2=0x7f565806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5658068380 0x7f565806a840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5664006e90 tx=0x7f5664008040 comp rx=0 tx=0).stop 2024-04-27T07:04:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 -- 172.21.15.12:0/57735238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56740b5ae0 msgr2=0x7f567413eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56740b5ae0 0x7f567413eaf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f56700010f0 tx=0x7f567000c890 comp rx=0 tx=0).stop 2024-04-27T07:04:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 -- 172.21.15.12:0/57735238 shutdown_connections 2024-04-27T07:04:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5658068380 0x7f565806a840 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5674148810 0x7f56741e0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56740b64c0 0x7f56741e0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 --2- 172.21.15.12:0/57735238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56740b5ae0 0x7f567413eaf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 -- 172.21.15.12:0/57735238 >> 172.21.15.12:0/57735238 conn(0x7f5674073c00 msgr2=0x7f5674140be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:18.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 -- 172.21.15.12:0/57735238 shutdown_connections 2024-04-27T07:04:18.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:18.989+0000 7f567c481640 1 -- 172.21.15.12:0/57735238 wait complete. 2024-04-27T07:04:19.011 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58ba436640 1 Processor -- start 2024-04-27T07:04:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58ba436640 1 -- start start 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58ba436640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f58b40b9a10 0x7f58b40b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58ba436640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 0x7f58b4145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58ba436640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58b406dac0 con 0x7f58b41460b0 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58ba436640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f58b406e890 con 0x7f58b40b9a10 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58ba436640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f58b40b9df0 con 0x7f58b40b3680 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58b3fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f58b40b9a10 0x7f58b40b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58b89d3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f58b41460b0 0x7f58b41490e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40802/0 (socket says 172.21.15.12:40802) 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58b89d3640 1 -- 172.21.15.12:0/388056739 learned_addr learned my addr 172.21.15.12:0/388056739 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58abfff640 1 --2- 172.21.15.12:0/388056739 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 0x7f58b4145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58b3fff640 1 -- 172.21.15.12:0/388056739 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 msgr2=0x7f58b4145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58b3fff640 1 --2- 172.21.15.12:0/388056739 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 0x7f58b4145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58b3fff640 1 -- 172.21.15.12:0/388056739 >> v1:172.21.15.12:6789/0 conn(0x7f58b41460b0 legacy=0x7f58b41490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58b3fff640 1 -- 172.21.15.12:0/388056739 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b406e090 con 0x7f58b40b9a10 2024-04-27T07:04:19.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58b37fe640 1 -- 172.21.15.12:0/388056739 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3626540236 0 0) 0x7f58ac0014a0 con 0x7f58b41460b0 2024-04-27T07:04:19.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58abfff640 1 --2- 172.21.15.12:0/388056739 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 0x7f58b4145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:19.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.133+0000 7f58b3fff640 1 --2- 172.21.15.12:0/388056739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b40b9a10 0x7f58b40b65c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58a40097c0 tx=0x7f58a401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d3bd5b4b1cf11a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:19.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b37fe640 1 -- 172.21.15.12:0/388056739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a40b0070 con 0x7f58b40b9a10 2024-04-27T07:04:19.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b37fe640 1 -- 172.21.15.12:0/388056739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58a40ab600 con 0x7f58b40b9a10 2024-04-27T07:04:19.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b37fe640 1 -- 172.21.15.12:0/388056739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a40ab9d0 con 0x7f58b40b9a10 2024-04-27T07:04:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- 172.21.15.12:0/388056739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b40b9a10 msgr2=0x7f58b40b65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 --2- 172.21.15.12:0/388056739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b40b9a10 0x7f58b40b65c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58a40097c0 tx=0x7f58a401c410 comp rx=0 tx=0).stop 2024-04-27T07:04:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- 172.21.15.12:0/388056739 shutdown_connections 2024-04-27T07:04:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 --2- 172.21.15.12:0/388056739 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 0x7f58b4145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 --2- 172.21.15.12:0/388056739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b40b9a10 0x7f58b40b65c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- 172.21.15.12:0/388056739 >> 172.21.15.12:0/388056739 conn(0x7f58b4067d10 msgr2=0x7f58b4068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- 172.21.15.12:0/388056739 shutdown_connections 2024-04-27T07:04:19.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- 172.21.15.12:0/388056739 wait complete. 2024-04-27T07:04:19.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 Processor -- start 2024-04-27T07:04:19.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- start start 2024-04-27T07:04:19.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 0x7f58b413c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58b40b9a10 0x7f58b41e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b41460b0 0x7f58b4137720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40b6b00 con 0x7f58b41460b0 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58b40740b0 con 0x7f58b40b9a10 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f58b40b6c80 con 0x7f58b40b3680 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b89d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b41460b0 0x7f58b4137720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b89d3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b41460b0 0x7f58b4137720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41892/0 (socket says 172.21.15.12:41892) 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b89d3640 1 -- 172.21.15.12:0/1650162912 learned_addr learned my addr 172.21.15.12:0/1650162912 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b3fff640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 0x7f58b413c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58abfff640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58b40b9a10 0x7f58b41e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b89d3640 1 -- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 msgr2=0x7f58b413c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b89d3640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 0x7f58b413c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b89d3640 1 -- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58b40b9a10 msgr2=0x7f58b41e17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b89d3640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58b40b9a10 0x7f58b41e17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b89d3640 1 -- 172.21.15.12:0/1650162912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b4139540 con 0x7f58b41460b0 2024-04-27T07:04:19.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b3fff640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 0x7f58b413c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b89d3640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b41460b0 0x7f58b4137720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00caf0 tx=0x7f58ac0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b17fa640 1 -- 172.21.15.12:0/1650162912 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00ac00 con 0x7f58b41460b0 2024-04-27T07:04:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b17fa640 1 -- 172.21.15.12:0/1650162912 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac00ada0 con 0x7f58b41460b0 2024-04-27T07:04:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- 172.21.15.12:0/1650162912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b413f180 con 0x7f58b41460b0 2024-04-27T07:04:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58b17fa640 1 -- 172.21.15.12:0/1650162912 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac012640 con 0x7f58b41460b0 2024-04-27T07:04:19.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.137+0000 7f58ba436640 1 -- 172.21.15.12:0/1650162912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b413ff60 con 0x7f58b41460b0 2024-04-27T07:04:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.141+0000 7f58ba436640 1 -- 172.21.15.12:0/1650162912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58b4068c20 con 0x7f58b41460b0 2024-04-27T07:04:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.141+0000 7f58b17fa640 1 -- 172.21.15.12:0/1650162912 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f58ac0127e0 con 0x7f58b41460b0 2024-04-27T07:04:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.141+0000 7f58b17fa640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f58940683b0 0x7f589406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.141+0000 7f58b17fa640 1 -- 172.21.15.12:0/1650162912 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58ac086f80 con 0x7f58b41460b0 2024-04-27T07:04:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.141+0000 7f58b3fff640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f58940683b0 0x7f589406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.141+0000 7f58b3fff640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f58940683b0 0x7f589406a870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58a40ad000 tx=0x7f58a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:19.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.145+0000 7f58b17fa640 1 -- 172.21.15.12:0/1650162912 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f58b406dac0 con 0x7f58b41460b0 2024-04-27T07:04:19.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.305+0000 7f58ba436640 1 -- 172.21.15.12:0/1650162912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58b406d7f0 con 0x7f58b41460b0 2024-04-27T07:04:19.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.309+0000 7f58b17fa640 1 -- 172.21.15.12:0/1650162912 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f58ac057540 con 0x7f58b41460b0 2024-04-27T07:04:19.314 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:19.314 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:19.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.309+0000 7f58ba436640 1 -- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f58940683b0 msgr2=0x7f589406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.309+0000 7f58ba436640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f58940683b0 0x7f589406a870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58a40ad000 tx=0x7f58a4008040 comp rx=0 tx=0).stop 2024-04-27T07:04:19.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.309+0000 7f58ba436640 1 -- 172.21.15.12:0/1650162912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b41460b0 msgr2=0x7f58b4137720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.309+0000 7f58ba436640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b41460b0 0x7f58b4137720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00caf0 tx=0x7f58ac0021d0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.313+0000 7f58ba436640 1 -- 172.21.15.12:0/1650162912 shutdown_connections 2024-04-27T07:04:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.313+0000 7f58ba436640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f58940683b0 0x7f589406a870 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.313+0000 7f58ba436640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58b41460b0 0x7f58b4137720 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.313+0000 7f58ba436640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58b40b9a10 0x7f58b41e17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.313+0000 7f58ba436640 1 --2- 172.21.15.12:0/1650162912 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f58b40b3680 0x7f58b413c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.313+0000 7f58ba436640 1 -- 172.21.15.12:0/1650162912 >> 172.21.15.12:0/1650162912 conn(0x7f58b4067d10 msgr2=0x7f58b40b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.313+0000 7f58ba436640 1 -- 172.21.15.12:0/1650162912 shutdown_connections 2024-04-27T07:04:19.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.313+0000 7f58ba436640 1 -- 172.21.15.12:0/1650162912 wait complete. 2024-04-27T07:04:19.333 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-27T07:04:19.333 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:04:19.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.437+0000 7f94277fe640 1 Processor -- start 2024-04-27T07:04:19.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.437+0000 7f94277fe640 1 -- start start 2024-04-27T07:04:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f94200b64c0 0x7f94200b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f94200b5ae0 0x7f9420148810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9420141ad0 0x7f9420141f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f94200b77d0 con 0x7f94200b5ae0 2024-04-27T07:04:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f94200b7560 con 0x7f94200b64c0 2024-04-27T07:04:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f94200b68a0 con 0x7f9420141ad0 2024-04-27T07:04:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9424d99640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f94200b5ae0 0x7f9420148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9424d99640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f94200b5ae0 0x7f9420148810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41898/0 (socket says 172.21.15.12:41898) 2024-04-27T07:04:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9424d99640 1 -- 172.21.15.12:0/2080719552 learned_addr learned my addr 172.21.15.12:0/2080719552 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:19.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9425d9b640 1 --2- 172.21.15.12:0/2080719552 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9420141ad0 0x7f9420141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f942559a640 1 --2- 172.21.15.12:0/2080719552 >> v2:172.21.15.53:3300/0 conn(0x7f94200b64c0 0x7f94200b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9424d99640 1 -- 172.21.15.12:0/2080719552 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9420141ad0 msgr2=0x7f9420141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:19.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9424d99640 1 --2- 172.21.15.12:0/2080719552 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9420141ad0 0x7f9420141f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9424d99640 1 -- 172.21.15.12:0/2080719552 >> v2:172.21.15.53:3300/0 conn(0x7f94200b64c0 msgr2=0x7f94200b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:19.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9424d99640 1 --2- 172.21.15.12:0/2080719552 >> v2:172.21.15.53:3300/0 conn(0x7f94200b64c0 0x7f94200b7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9424d99640 1 -- 172.21.15.12:0/2080719552 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9420148010 con 0x7f94200b5ae0 2024-04-27T07:04:19.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9425d9b640 1 --2- 172.21.15.12:0/2080719552 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9420141ad0 0x7f9420141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:19.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f942559a640 1 --2- 172.21.15.12:0/2080719552 >> v2:172.21.15.53:3300/0 conn(0x7f94200b64c0 0x7f94200b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:19.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9424d99640 1 --2- 172.21.15.12:0/2080719552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94200b5ae0 0x7f9420148810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f94200be1f0 tx=0x7f941001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38a3320fe1f5f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:19.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9417fff640 1 -- 172.21.15.12:0/2080719552 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94100b0070 con 0x7f94200b5ae0 2024-04-27T07:04:19.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9417fff640 1 -- 172.21.15.12:0/2080719552 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94100ab600 con 0x7f94200b5ae0 2024-04-27T07:04:19.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f9417fff640 1 -- 172.21.15.12:0/2080719552 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94100ab9d0 con 0x7f94200b5ae0 2024-04-27T07:04:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 -- 172.21.15.12:0/2080719552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94200b5ae0 msgr2=0x7f9420148810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 --2- 172.21.15.12:0/2080719552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94200b5ae0 0x7f9420148810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f94200be1f0 tx=0x7f941001c400 comp rx=0 tx=0).stop 2024-04-27T07:04:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 -- 172.21.15.12:0/2080719552 shutdown_connections 2024-04-27T07:04:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 --2- 172.21.15.12:0/2080719552 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9420141ad0 0x7f9420141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 --2- 172.21.15.12:0/2080719552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94200b5ae0 0x7f9420148810 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 --2- 172.21.15.12:0/2080719552 >> v2:172.21.15.53:3300/0 conn(0x7f94200b64c0 0x7f94200b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 -- 172.21.15.12:0/2080719552 >> 172.21.15.12:0/2080719552 conn(0x7f9420073c00 msgr2=0x7f94200bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 -- 172.21.15.12:0/2080719552 shutdown_connections 2024-04-27T07:04:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.441+0000 7f94277fe640 1 -- 172.21.15.12:0/2080719552 wait complete. 2024-04-27T07:04:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 Processor -- start 2024-04-27T07:04:19.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 -- start start 2024-04-27T07:04:19.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94200b5ae0 0x7f94201e0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94200b64c0 0x7f9420139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9420141ad0 0x7f942013f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f942014bc40 con 0x7f9420141ad0 2024-04-27T07:04:19.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f942014bac0 con 0x7f94200b64c0 2024-04-27T07:04:19.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f942014bdc0 con 0x7f94200b5ae0 2024-04-27T07:04:19.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9425d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9420141ad0 0x7f942013f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9425d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9420141ad0 0x7f942013f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41914/0 (socket says 172.21.15.12:41914) 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9425d9b640 1 -- 172.21.15.12:0/1949895144 learned_addr learned my addr 172.21.15.12:0/1949895144 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f942559a640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94200b5ae0 0x7f94201e0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9424d99640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94200b64c0 0x7f9420139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9425d9b640 1 -- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94200b5ae0 msgr2=0x7f94201e0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9425d9b640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94200b5ae0 0x7f94201e0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9425d9b640 1 -- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94200b64c0 msgr2=0x7f9420139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9425d9b640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94200b64c0 0x7f9420139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9425d9b640 1 -- 172.21.15.12:0/1949895144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94201381c0 con 0x7f9420141ad0 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f942559a640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94200b5ae0 0x7f94201e0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9424d99640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94200b64c0 0x7f9420139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9425d9b640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9420141ad0 0x7f942013f410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f941800cd50 tx=0x7f941800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9415ffb640 1 -- 172.21.15.12:0/1949895144 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941800f680 con 0x7f9420141ad0 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9415ffb640 1 -- 172.21.15.12:0/1949895144 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f941800f840 con 0x7f9420141ad0 2024-04-27T07:04:19.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 -- 172.21.15.12:0/1949895144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94201395e0 con 0x7f9420141ad0 2024-04-27T07:04:19.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9415ffb640 1 -- 172.21.15.12:0/1949895144 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941800fc30 con 0x7f9420141ad0 2024-04-27T07:04:19.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 -- 172.21.15.12:0/1949895144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942013b2d0 con 0x7f9420141ad0 2024-04-27T07:04:19.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f94277fe640 1 -- 172.21.15.12:0/1949895144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9420068c20 con 0x7f9420141ad0 2024-04-27T07:04:19.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.445+0000 7f9415ffb640 1 -- 172.21.15.12:0/1949895144 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f94180213f0 con 0x7f9420141ad0 2024-04-27T07:04:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.449+0000 7f9415ffb640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9404068380 0x7f940406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.449+0000 7f9415ffb640 1 -- 172.21.15.12:0/1949895144 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9418087630 con 0x7f9420141ad0 2024-04-27T07:04:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.449+0000 7f942559a640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9404068380 0x7f940406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.449+0000 7f942559a640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9404068380 0x7f940406a840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f941c008930 tx=0x7f941c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:19.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.453+0000 7f9415ffb640 1 -- 172.21.15.12:0/1949895144 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9418014030 con 0x7f9420141ad0 2024-04-27T07:04:19.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.601+0000 7f94277fe640 1 -- 172.21.15.12:0/1949895144 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9420138390 con 0x7f9404068380 2024-04-27T07:04:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.621+0000 7f9415ffb640 1 -- 172.21.15.12:0/1949895144 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240067 (secure 0 0 0) 0x7f9420138390 con 0x7f9404068380 2024-04-27T07:04:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:19.630 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 -- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9404068380 msgr2=0x7f940406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9404068380 0x7f940406a840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f941c008930 tx=0x7f941c008040 comp rx=0 tx=0).stop 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 -- 172.21.15.12:0/1949895144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9420141ad0 msgr2=0x7f942013f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9420141ad0 0x7f942013f410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f941800cd50 tx=0x7f941800a650 comp rx=0 tx=0).stop 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 -- 172.21.15.12:0/1949895144 shutdown_connections 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9404068380 0x7f940406a840 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9420141ad0 0x7f942013f410 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94200b64c0 0x7f9420139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 --2- 172.21.15.12:0/1949895144 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94200b5ae0 0x7f94201e0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 -- 172.21.15.12:0/1949895144 >> 172.21.15.12:0/1949895144 conn(0x7f9420073c00 msgr2=0x7f94200ba600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:19.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 -- 172.21.15.12:0/1949895144 shutdown_connections 2024-04-27T07:04:19.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.625+0000 7f94277fe640 1 -- 172.21.15.12:0/1949895144 wait complete. 2024-04-27T07:04:19.645 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-27T07:04:18.609036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235236,"kb_used_data":2424,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739484,"statfs":{"total":773094113280,"available":772853231616,"internally_reserved":0,"allocated":2482176,"data_stored":1570140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030242"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.576639+0000","last_change":"2024-04-27T07:04:13.576639+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.576639+0000","last_undegraded":"2024-04-27T07:04:13.576639+0000","last_fullsized":"2024-04-27T07:04:13.576639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.585621+0000","last_change":"2024-04-27T07:04:13.585621+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.585621+0000","last_undegraded":"2024-04-27T07:04:13.585621+0000","last_fullsized":"2024-04-27T07:04:13.585621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.577307+0000","last_change":"2024-04-27T07:04:13.577307+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.577307+0000","last_undegraded":"2024-04-27T07:04:13.577307+0000","last_fullsized":"2024-04-27T07:04:13.577307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579242+0000","last_change":"2024-04-27T07:04:13.579242+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579242+0000","last_undegraded":"2024-04-27T07:04:13.579242+0000","last_fullsized":"2024-04-27T07:04:13.579242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.588397+0000","last_change":"2024-04-27T07:04:13.588397+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.588397+0000","last_undegraded":"2024-04-27T07:04:13.588397+0000","last_fullsized":"2024-04-27T07:04:13.588397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.583927+0000","last_change":"2024-04-27T07:04:13.583927+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.583927+0000","last_undegraded":"2024-04-27T07:04:13.583927+0000","last_fullsized":"2024-04-27T07:04:13.583927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.580842+0000","last_change":"2024-04-27T07:04:13.580842+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.580842+0000","last_undegraded":"2024-04-27T07:04:13.580842+0000","last_fullsized":"2024-04-27T07:04:13.580842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.586516+0000","last_change":"2024-04-27T07:04:13.586516+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.586516+0000","last_undegraded":"2024-04-27T07:04:13.586516+0000","last_fullsized":"2024-04-27T07:04:13.586516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.577547+0000","last_change":"2024-04-27T07:04:13.577547+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.577547+0000","last_undegraded":"2024-04-27T07:04:13.577547+0000","last_fullsized":"2024-04-27T07:04:13.577547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.584610+0000","last_change":"2024-04-27T07:04:13.584610+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.584610+0000","last_undegraded":"2024-04-27T07:04:13.584610+0000","last_fullsized":"2024-04-27T07:04:13.584610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.587198+0000","last_change":"2024-04-27T07:04:13.587198+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.587198+0000","last_undegraded":"2024-04-27T07:04:13.587198+0000","last_fullsized":"2024-04-27T07:04:13.587198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.582193+0000","last_change":"2024-04-27T07:04:13.582193+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.582193+0000","last_undegraded":"2024-04-27T07:04:13.582193+0000","last_fullsized":"2024-04-27T07:04:13.582193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573203+0000","last_change":"2024-04-27T07:04:13.573203+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573203+0000","last_undegraded":"2024-04-27T07:04:13.573203+0000","last_fullsized":"2024-04-27T07:04:13.573203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.582149+0000","last_change":"2024-04-27T07:04:13.582149+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.582149+0000","last_undegraded":"2024-04-27T07:04:13.582149+0000","last_fullsized":"2024-04-27T07:04:13.582149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579869+0000","last_change":"2024-04-27T07:04:13.579869+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579869+0000","last_undegraded":"2024-04-27T07:04:13.579869+0000","last_fullsized":"2024-04-27T07:04:13.579869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.583247+0000","last_change":"2024-04-27T07:04:13.583247+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.583247+0000","last_undegraded":"2024-04-27T07:04:13.583247+0000","last_fullsized":"2024-04-27T07:04:13.583247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-27T07:04:13.576472+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:04:13.576472+0000","last_peered":"2024-04-27T07:04:13.576472+0000","last_clean":"2024-04-27T07:04:13.576472+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:04:13.576472+0000","last_undegraded":"2024-04-27T07:04:13.576472+0000","last_fullsized":"2024-04-27T07:04:13.576472+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.578258+0000","last_change":"2024-04-27T07:04:13.578258+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.578258+0000","last_undegraded":"2024-04-27T07:04:13.578258+0000","last_fullsized":"2024-04-27T07:04:13.578258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573422+0000","last_change":"2024-04-27T07:04:13.573422+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573422+0000","last_undegraded":"2024-04-27T07:04:13.573422+0000","last_fullsized":"2024-04-27T07:04:13.573422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.582903+0000","last_change":"2024-04-27T07:04:13.582903+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.582903+0000","last_undegraded":"2024-04-27T07:04:13.582903+0000","last_fullsized":"2024-04-27T07:04:13.582903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579967+0000","last_change":"2024-04-27T07:04:13.579967+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579967+0000","last_undegraded":"2024-04-27T07:04:13.579967+0000","last_fullsized":"2024-04-27T07:04:13.579967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579686+0000","last_change":"2024-04-27T07:04:13.579686+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579686+0000","last_undegraded":"2024-04-27T07:04:13.579686+0000","last_fullsized":"2024-04-27T07:04:13.579686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573037+0000","last_change":"2024-04-27T07:04:13.573037+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573037+0000","last_undegraded":"2024-04-27T07:04:13.573037+0000","last_fullsized":"2024-04-27T07:04:13.573037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.581604+0000","last_change":"2024-04-27T07:04:13.581604+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.581604+0000","last_undegraded":"2024-04-27T07:04:13.581604+0000","last_fullsized":"2024-04-27T07:04:13.581604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.587674+0000","last_change":"2024-04-27T07:04:13.587674+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.587674+0000","last_undegraded":"2024-04-27T07:04:13.587674+0000","last_fullsized":"2024-04-27T07:04:13.587674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.574827+0000","last_change":"2024-04-27T07:04:13.574827+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.574827+0000","last_undegraded":"2024-04-27T07:04:13.574827+0000","last_fullsized":"2024-04-27T07:04:13.574827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.577590+0000","last_change":"2024-04-27T07:04:13.577590+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.577590+0000","last_undegraded":"2024-04-27T07:04:13.577590+0000","last_fullsized":"2024-04-27T07:04:13.577590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.578559+0000","last_change":"2024-04-27T07:04:13.578559+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.578559+0000","last_undegraded":"2024-04-27T07:04:13.578559+0000","last_fullsized":"2024-04-27T07:04:13.578559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573457+0000","last_change":"2024-04-27T07:04:13.573457+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573457+0000","last_undegraded":"2024-04-27T07:04:13.573457+0000","last_fullsized":"2024-04-27T07:04:13.573457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573722+0000","last_change":"2024-04-27T07:04:13.573722+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573722+0000","last_undegraded":"2024-04-27T07:04:13.573722+0000","last_fullsized":"2024-04-27T07:04:13.573722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.575544+0000","last_change":"2024-04-27T07:04:13.575544+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.575544+0000","last_undegraded":"2024-04-27T07:04:13.575544+0000","last_fullsized":"2024-04-27T07:04:13.575544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.576708+0000","last_change":"2024-04-27T07:04:13.576708+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.576708+0000","last_undegraded":"2024-04-27T07:04:13.576708+0000","last_fullsized":"2024-04-27T07:04:13.576708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.578313+0000","last_change":"2024-04-27T07:04:13.578313+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.578313+0000","last_undegraded":"2024-04-27T07:04:13.578313+0000","last_fullsized":"2024-04-27T07:04:13.578313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:04:19.650 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:04:19.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3ec2b4640 1 Processor -- start 2024-04-27T07:04:19.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3ec2b4640 1 -- start start 2024-04-27T07:04:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3ec2b4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3e40b62a0 0x7fc3e40b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3ec2b4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc3e41485b0 0x7fc3e414b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3ec2b4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3e4147db0 con 0x7fc3e4148a30 2024-04-27T07:04:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3ec2b4640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fc3e4148090 con 0x7fc3e41485b0 2024-04-27T07:04:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3ec2b4640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e414c170 con 0x7fc3e40b62a0 2024-04-27T07:04:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3e984f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc3e41485b0 0x7fc3e414b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3e984f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc3e41485b0 0x7fc3e414b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41930/0 (socket says 172.21.15.12:41930) 2024-04-27T07:04:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3e984f640 1 -- 172.21.15.12:0/3745192993 learned_addr learned my addr 172.21.15.12:0/3745192993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3ea050640 1 --2- 172.21.15.12:0/3745192993 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3e40b62a0 0x7fc3e40b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3e984f640 1 -- 172.21.15.12:0/3745192993 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3e40b62a0 msgr2=0x7fc3e40b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:19.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3e984f640 1 --2- 172.21.15.12:0/3745192993 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3e40b62a0 0x7fc3e40b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3e984f640 1 -- 172.21.15.12:0/3745192993 >> v1:172.21.15.53:6789/0 conn(0x7fc3e4148a30 legacy=0x7fc3e4141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:19.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.801+0000 7fc3e984f640 1 -- 172.21.15.12:0/3745192993 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e40b7f60 con 0x7fc3e41485b0 2024-04-27T07:04:19.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3e984f640 1 --2- 172.21.15.12:0/3745192993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e41485b0 0x7fc3e414b770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009870 tx=0x7fc3d801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a0b9c90d87e1cf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:19.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3e904e640 1 -- 172.21.15.12:0/3745192993 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80a7070 con 0x7fc3e41485b0 2024-04-27T07:04:19.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3e904e640 1 -- 172.21.15.12:0/3745192993 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d80b0070 con 0x7fc3e41485b0 2024-04-27T07:04:19.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3e904e640 1 -- 172.21.15.12:0/3745192993 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80ab890 con 0x7fc3e41485b0 2024-04-27T07:04:19.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/3745192993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e41485b0 msgr2=0x7fc3e414b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 --2- 172.21.15.12:0/3745192993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e41485b0 0x7fc3e414b770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8009870 tx=0x7fc3d801c490 comp rx=0 tx=0).stop 2024-04-27T07:04:19.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/3745192993 shutdown_connections 2024-04-27T07:04:19.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 --2- 172.21.15.12:0/3745192993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e41485b0 0x7fc3e414b770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 --2- 172.21.15.12:0/3745192993 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3e40b62a0 0x7fc3e40b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/3745192993 >> 172.21.15.12:0/3745192993 conn(0x7fc3e4068190 msgr2=0x7fc3e40bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:19.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/3745192993 shutdown_connections 2024-04-27T07:04:19.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/3745192993 wait complete. 2024-04-27T07:04:19.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 Processor -- start 2024-04-27T07:04:19.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 -- start start 2024-04-27T07:04:19.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3e40b62a0 0x7fc3e413f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41485b0 0x7fc3e41e3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e4148a30 0x7fc3e41e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e41551d0 con 0x7fc3e4148a30 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e4155050 con 0x7fc3e41485b0 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.805+0000 7fc3ec2b4640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e4155350 con 0x7fc3e40b62a0 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea851640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e4148a30 0x7fc3e41e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea851640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e4148a30 0x7fc3e41e44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41944/0 (socket says 172.21.15.12:41944) 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea851640 1 -- 172.21.15.12:0/1247567539 learned_addr learned my addr 172.21.15.12:0/1247567539 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea050640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3e40b62a0 0x7fc3e413f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3e984f640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41485b0 0x7fc3e41e3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea851640 1 -- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3e40b62a0 msgr2=0x7fc3e413f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea851640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3e40b62a0 0x7fc3e413f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea851640 1 -- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41485b0 msgr2=0x7fc3e41e3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea851640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41485b0 0x7fc3e41e3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea851640 1 -- 172.21.15.12:0/1247567539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e41e4a10 con 0x7fc3e4148a30 2024-04-27T07:04:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea851640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e4148a30 0x7fc3e41e44d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400c390 tx=0x7fc3d400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3e2ffd640 1 -- 172.21.15.12:0/1247567539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4011790 con 0x7fc3e4148a30 2024-04-27T07:04:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3e2ffd640 1 -- 172.21.15.12:0/1247567539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d4017070 con 0x7fc3e4148a30 2024-04-27T07:04:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/1247567539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e40bb3c0 con 0x7fc3e4148a30 2024-04-27T07:04:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3e2ffd640 1 -- 172.21.15.12:0/1247567539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4011b70 con 0x7fc3e4148a30 2024-04-27T07:04:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/1247567539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e41409f0 con 0x7fc3e4148a30 2024-04-27T07:04:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/1247567539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e4069330 con 0x7fc3e4148a30 2024-04-27T07:04:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3e2ffd640 1 -- 172.21.15.12:0/1247567539 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc3d4011d10 con 0x7fc3e4148a30 2024-04-27T07:04:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3e2ffd640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc3d0068380 0x7fc3d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3e2ffd640 1 -- 172.21.15.12:0/1247567539 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc3d40864c0 con 0x7fc3e4148a30 2024-04-27T07:04:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea050640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc3d0068380 0x7fc3d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.809+0000 7fc3ea050640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc3d0068380 0x7fc3d006a840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0070b0 tx=0x7fc3dc007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.813+0000 7fc3e2ffd640 1 -- 172.21.15.12:0/1247567539 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc3d4021020 con 0x7fc3e4148a30 2024-04-27T07:04:19.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.973+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/1247567539 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3e40bd940 con 0x7fc3d0068380 2024-04-27T07:04:19.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.989+0000 7fc3e2ffd640 1 -- 172.21.15.12:0/1247567539 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240067 (secure 0 0 0) 0x7fc3e40bd940 con 0x7fc3d0068380 2024-04-27T07:04:19.996 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:19.997 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:04:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc3d0068380 msgr2=0x7fc3d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc3d0068380 0x7fc3d006a840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0070b0 tx=0x7fc3dc007040 comp rx=0 tx=0).stop 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/1247567539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e4148a30 msgr2=0x7fc3e41e44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e4148a30 0x7fc3e41e44d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400c390 tx=0x7fc3d400c850 comp rx=0 tx=0).stop 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/1247567539 shutdown_connections 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc3d0068380 0x7fc3d006a840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc3e4148a30 0x7fc3e41e44d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc3e41485b0 0x7fc3e41e3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 --2- 172.21.15.12:0/1247567539 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc3e40b62a0 0x7fc3e413f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/1247567539 >> 172.21.15.12:0/1247567539 conn(0x7fc3e4068190 msgr2=0x7fc3e40b8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/1247567539 shutdown_connections 2024-04-27T07:04:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:19.993+0000 7fc3ec2b4640 1 -- 172.21.15.12:0/1247567539 wait complete. 2024-04-27T07:04:20.016 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-27T07:04:18.609036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235236,"kb_used_data":2424,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739484,"statfs":{"total":773094113280,"available":772853231616,"internally_reserved":0,"allocated":2482176,"data_stored":1570140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030242"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.576639+0000","last_change":"2024-04-27T07:04:13.576639+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.576639+0000","last_undegraded":"2024-04-27T07:04:13.576639+0000","last_fullsized":"2024-04-27T07:04:13.576639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.585621+0000","last_change":"2024-04-27T07:04:13.585621+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.585621+0000","last_undegraded":"2024-04-27T07:04:13.585621+0000","last_fullsized":"2024-04-27T07:04:13.585621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.577307+0000","last_change":"2024-04-27T07:04:13.577307+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.577307+0000","last_undegraded":"2024-04-27T07:04:13.577307+0000","last_fullsized":"2024-04-27T07:04:13.577307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579242+0000","last_change":"2024-04-27T07:04:13.579242+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579242+0000","last_undegraded":"2024-04-27T07:04:13.579242+0000","last_fullsized":"2024-04-27T07:04:13.579242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.588397+0000","last_change":"2024-04-27T07:04:13.588397+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.588397+0000","last_undegraded":"2024-04-27T07:04:13.588397+0000","last_fullsized":"2024-04-27T07:04:13.588397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.583927+0000","last_change":"2024-04-27T07:04:13.583927+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.583927+0000","last_undegraded":"2024-04-27T07:04:13.583927+0000","last_fullsized":"2024-04-27T07:04:13.583927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.580842+0000","last_change":"2024-04-27T07:04:13.580842+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.580842+0000","last_undegraded":"2024-04-27T07:04:13.580842+0000","last_fullsized":"2024-04-27T07:04:13.580842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.586516+0000","last_change":"2024-04-27T07:04:13.586516+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.586516+0000","last_undegraded":"2024-04-27T07:04:13.586516+0000","last_fullsized":"2024-04-27T07:04:13.586516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.577547+0000","last_change":"2024-04-27T07:04:13.577547+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.577547+0000","last_undegraded":"2024-04-27T07:04:13.577547+0000","last_fullsized":"2024-04-27T07:04:13.577547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.584610+0000","last_change":"2024-04-27T07:04:13.584610+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.584610+0000","last_undegraded":"2024-04-27T07:04:13.584610+0000","last_fullsized":"2024-04-27T07:04:13.584610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.587198+0000","last_change":"2024-04-27T07:04:13.587198+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.587198+0000","last_undegraded":"2024-04-27T07:04:13.587198+0000","last_fullsized":"2024-04-27T07:04:13.587198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.582193+0000","last_change":"2024-04-27T07:04:13.582193+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.582193+0000","last_undegraded":"2024-04-27T07:04:13.582193+0000","last_fullsized":"2024-04-27T07:04:13.582193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573203+0000","last_change":"2024-04-27T07:04:13.573203+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573203+0000","last_undegraded":"2024-04-27T07:04:13.573203+0000","last_fullsized":"2024-04-27T07:04:13.573203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.582149+0000","last_change":"2024-04-27T07:04:13.582149+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.582149+0000","last_undegraded":"2024-04-27T07:04:13.582149+0000","last_fullsized":"2024-04-27T07:04:13.582149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579869+0000","last_change":"2024-04-27T07:04:13.579869+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579869+0000","last_undegraded":"2024-04-27T07:04:13.579869+0000","last_fullsized":"2024-04-27T07:04:13.579869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.583247+0000","last_change":"2024-04-27T07:04:13.583247+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.583247+0000","last_undegraded":"2024-04-27T07:04:13.583247+0000","last_fullsized":"2024-04-27T07:04:13.583247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-27T07:04:13.576472+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:04:13.576472+0000","last_peered":"2024-04-27T07:04:13.576472+0000","last_clean":"2024-04-27T07:04:13.576472+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:04:13.576472+0000","last_undegraded":"2024-04-27T07:04:13.576472+0000","last_fullsized":"2024-04-27T07:04:13.576472+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.578258+0000","last_change":"2024-04-27T07:04:13.578258+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.578258+0000","last_undegraded":"2024-04-27T07:04:13.578258+0000","last_fullsized":"2024-04-27T07:04:13.578258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573422+0000","last_change":"2024-04-27T07:04:13.573422+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573422+0000","last_undegraded":"2024-04-27T07:04:13.573422+0000","last_fullsized":"2024-04-27T07:04:13.573422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.582903+0000","last_change":"2024-04-27T07:04:13.582903+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.582903+0000","last_undegraded":"2024-04-27T07:04:13.582903+0000","last_fullsized":"2024-04-27T07:04:13.582903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579967+0000","last_change":"2024-04-27T07:04:13.579967+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579967+0000","last_undegraded":"2024-04-27T07:04:13.579967+0000","last_fullsized":"2024-04-27T07:04:13.579967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579686+0000","last_change":"2024-04-27T07:04:13.579686+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579686+0000","last_undegraded":"2024-04-27T07:04:13.579686+0000","last_fullsized":"2024-04-27T07:04:13.579686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573037+0000","last_change":"2024-04-27T07:04:13.573037+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573037+0000","last_undegraded":"2024-04-27T07:04:13.573037+0000","last_fullsized":"2024-04-27T07:04:13.573037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.581604+0000","last_change":"2024-04-27T07:04:13.581604+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.581604+0000","last_undegraded":"2024-04-27T07:04:13.581604+0000","last_fullsized":"2024-04-27T07:04:13.581604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.587674+0000","last_change":"2024-04-27T07:04:13.587674+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.587674+0000","last_undegraded":"2024-04-27T07:04:13.587674+0000","last_fullsized":"2024-04-27T07:04:13.587674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.574827+0000","last_change":"2024-04-27T07:04:13.574827+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.574827+0000","last_undegraded":"2024-04-27T07:04:13.574827+0000","last_fullsized":"2024-04-27T07:04:13.574827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.577590+0000","last_change":"2024-04-27T07:04:13.577590+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.577590+0000","last_undegraded":"2024-04-27T07:04:13.577590+0000","last_fullsized":"2024-04-27T07:04:13.577590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.578559+0000","last_change":"2024-04-27T07:04:13.578559+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.578559+0000","last_undegraded":"2024-04-27T07:04:13.578559+0000","last_fullsized":"2024-04-27T07:04:13.578559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573457+0000","last_change":"2024-04-27T07:04:13.573457+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573457+0000","last_undegraded":"2024-04-27T07:04:13.573457+0000","last_fullsized":"2024-04-27T07:04:13.573457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573722+0000","last_change":"2024-04-27T07:04:13.573722+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573722+0000","last_undegraded":"2024-04-27T07:04:13.573722+0000","last_fullsized":"2024-04-27T07:04:13.573722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.575544+0000","last_change":"2024-04-27T07:04:13.575544+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.575544+0000","last_undegraded":"2024-04-27T07:04:13.575544+0000","last_fullsized":"2024-04-27T07:04:13.575544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.576708+0000","last_change":"2024-04-27T07:04:13.576708+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.576708+0000","last_undegraded":"2024-04-27T07:04:13.576708+0000","last_fullsized":"2024-04-27T07:04:13.576708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.578313+0000","last_change":"2024-04-27T07:04:13.578313+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.578313+0000","last_undegraded":"2024-04-27T07:04:13.578313+0000","last_fullsized":"2024-04-27T07:04:13.578313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:04:20.020 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:04:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe735d0640 1 Processor -- start 2024-04-27T07:04:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe735d0640 1 -- start start 2024-04-27T07:04:20.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe735d0640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efe6c1487b0 0x7efe6c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe735d0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efe6c13e5b0 0x7efe6c13ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe735d0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6c0b9ec0 con 0x7efe6c0b9ae0 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe735d0640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7efe6c09ddb0 con 0x7efe6c1487b0 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe735d0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7efe6c09e090 con 0x7efe6c13e5b0 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe70b6b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efe6c1487b0 0x7efe6c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe70b6b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efe6c1487b0 0x7efe6c073c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41950/0 (socket says 172.21.15.12:41950) 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe70b6b640 1 -- 172.21.15.12:0/1590543475 learned_addr learned my addr 172.21.15.12:0/1590543475 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe71b6d640 1 --2- 172.21.15.12:0/1590543475 >> v2:172.21.15.53:3300/0 conn(0x7efe6c13e5b0 0x7efe6c13ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe70b6b640 1 -- 172.21.15.12:0/1590543475 >> v2:172.21.15.53:3300/0 conn(0x7efe6c13e5b0 msgr2=0x7efe6c13ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe70b6b640 1 --2- 172.21.15.12:0/1590543475 >> v2:172.21.15.53:3300/0 conn(0x7efe6c13e5b0 0x7efe6c13ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe70b6b640 1 -- 172.21.15.12:0/1590543475 >> v1:172.21.15.53:6789/0 conn(0x7efe6c0b9ae0 legacy=0x7efe6c1480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe70b6b640 1 -- 172.21.15.12:0/1590543475 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c13efa0 con 0x7efe6c1487b0 2024-04-27T07:04:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.177+0000 7efe70b6b640 1 --2- 172.21.15.12:0/1590543475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 0x7efe6c073c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009870 tx=0x7efe5c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56df2b564fb12a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe63fff640 1 -- 172.21.15.12:0/1590543475 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0b0070 con 0x7efe6c1487b0 2024-04-27T07:04:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe63fff640 1 -- 172.21.15.12:0/1590543475 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c0ab600 con 0x7efe6c1487b0 2024-04-27T07:04:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe63fff640 1 -- 172.21.15.12:0/1590543475 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0ab9d0 con 0x7efe6c1487b0 2024-04-27T07:04:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- 172.21.15.12:0/1590543475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 msgr2=0x7efe6c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 --2- 172.21.15.12:0/1590543475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 0x7efe6c073c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009870 tx=0x7efe5c01c4e0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- 172.21.15.12:0/1590543475 shutdown_connections 2024-04-27T07:04:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 --2- 172.21.15.12:0/1590543475 >> v2:172.21.15.53:3300/0 conn(0x7efe6c13e5b0 0x7efe6c13ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 --2- 172.21.15.12:0/1590543475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 0x7efe6c073c60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- 172.21.15.12:0/1590543475 >> 172.21.15.12:0/1590543475 conn(0x7efe6c067d10 msgr2=0x7efe6c068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- 172.21.15.12:0/1590543475 shutdown_connections 2024-04-27T07:04:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- 172.21.15.12:0/1590543475 wait complete. 2024-04-27T07:04:20.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 Processor -- start 2024-04-27T07:04:20.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- start start 2024-04-27T07:04:20.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe6c0b9ae0 0x7efe6c13bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe6c13e5b0 0x7efe6c0b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 0x7efe6c1395d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c0b5110 con 0x7efe6c1487b0 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c09db90 con 0x7efe6c0b9ae0 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7efe6c151cb0 con 0x7efe6c13e5b0 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe71b6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 0x7efe6c1395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe71b6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 0x7efe6c1395d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41956/0 (socket says 172.21.15.12:41956) 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe71b6d640 1 -- 172.21.15.12:0/3395008940 learned_addr learned my addr 172.21.15.12:0/3395008940 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe70b6b640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe6c13e5b0 0x7efe6c0b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe7136c640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe6c0b9ae0 0x7efe6c13bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe71b6d640 1 -- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe6c13e5b0 msgr2=0x7efe6c0b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe71b6d640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe6c13e5b0 0x7efe6c0b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe71b6d640 1 -- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe6c0b9ae0 msgr2=0x7efe6c13bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe71b6d640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe6c0b9ae0 0x7efe6c13bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe71b6d640 1 -- 172.21.15.12:0/3395008940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c14e790 con 0x7efe6c1487b0 2024-04-27T07:04:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe70b6b640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe6c13e5b0 0x7efe6c0b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe7136c640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe6c0b9ae0 0x7efe6c13bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe71b6d640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 0x7efe6c1395d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efe64007e70 tx=0x7efe6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe61ffb640 1 -- 172.21.15.12:0/3395008940 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64014620 con 0x7efe6c1487b0 2024-04-27T07:04:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe61ffb640 1 -- 172.21.15.12:0/3395008940 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe64019070 con 0x7efe6c1487b0 2024-04-27T07:04:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe61ffb640 1 -- 172.21.15.12:0/3395008940 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64014a20 con 0x7efe6c1487b0 2024-04-27T07:04:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- 172.21.15.12:0/3395008940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c0b9110 con 0x7efe6c1487b0 2024-04-27T07:04:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.181+0000 7efe735d0640 1 -- 172.21.15.12:0/3395008940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c147500 con 0x7efe6c1487b0 2024-04-27T07:04:20.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.185+0000 7efe735d0640 1 -- 172.21.15.12:0/3395008940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6c068c20 con 0x7efe6c1487b0 2024-04-27T07:04:20.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.185+0000 7efe61ffb640 1 -- 172.21.15.12:0/3395008940 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efe64014bc0 con 0x7efe6c1487b0 2024-04-27T07:04:20.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.185+0000 7efe61ffb640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efe50068400 0x7efe5006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.185+0000 7efe61ffb640 1 -- 172.21.15.12:0/3395008940 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efe64087210 con 0x7efe6c1487b0 2024-04-27T07:04:20.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.185+0000 7efe7136c640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efe50068400 0x7efe5006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.185+0000 7efe7136c640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efe50068400 0x7efe5006a8c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efe6800a850 tx=0x7efe68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.189+0000 7efe61ffb640 1 -- 172.21.15.12:0/3395008940 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe64057840 con 0x7efe6c1487b0 2024-04-27T07:04:20.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.353+0000 7efe735d0640 1 -- 172.21.15.12:0/3395008940 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe6c0b5560 con 0x7efe50068400 2024-04-27T07:04:20.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe61ffb640 1 -- 172.21.15.12:0/3395008940 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240067 (secure 0 0 0) 0x7efe6c0b5560 con 0x7efe50068400 2024-04-27T07:04:20.378 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:20.380 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe735d0640 1 -- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efe50068400 msgr2=0x7efe5006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe735d0640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efe50068400 0x7efe5006a8c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efe6800a850 tx=0x7efe68008040 comp rx=0 tx=0).stop 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe735d0640 1 -- 172.21.15.12:0/3395008940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 msgr2=0x7efe6c1395d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe735d0640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 0x7efe6c1395d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efe64007e70 tx=0x7efe6400e4b0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe735d0640 1 -- 172.21.15.12:0/3395008940 shutdown_connections 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe735d0640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efe50068400 0x7efe5006a8c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe735d0640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6c1487b0 0x7efe6c1395d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe735d0640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efe6c13e5b0 0x7efe6c0b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe735d0640 1 --2- 172.21.15.12:0/3395008940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe6c0b9ae0 0x7efe6c13bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.373+0000 7efe735d0640 1 -- 172.21.15.12:0/3395008940 >> 172.21.15.12:0/3395008940 conn(0x7efe6c067d10 msgr2=0x7efe6c1480a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:20.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.377+0000 7efe735d0640 1 -- 172.21.15.12:0/3395008940 shutdown_connections 2024-04-27T07:04:20.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.377+0000 7efe735d0640 1 -- 172.21.15.12:0/3395008940 wait complete. 2024-04-27T07:04:20.398 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-27T07:04:18.609036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235236,"kb_used_data":2424,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739484,"statfs":{"total":773094113280,"available":772853231616,"internally_reserved":0,"allocated":2482176,"data_stored":1570140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030242"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.576639+0000","last_change":"2024-04-27T07:04:13.576639+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.576639+0000","last_undegraded":"2024-04-27T07:04:13.576639+0000","last_fullsized":"2024-04-27T07:04:13.576639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.585621+0000","last_change":"2024-04-27T07:04:13.585621+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.585621+0000","last_undegraded":"2024-04-27T07:04:13.585621+0000","last_fullsized":"2024-04-27T07:04:13.585621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.577307+0000","last_change":"2024-04-27T07:04:13.577307+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.577307+0000","last_undegraded":"2024-04-27T07:04:13.577307+0000","last_fullsized":"2024-04-27T07:04:13.577307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579242+0000","last_change":"2024-04-27T07:04:13.579242+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579242+0000","last_undegraded":"2024-04-27T07:04:13.579242+0000","last_fullsized":"2024-04-27T07:04:13.579242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.588397+0000","last_change":"2024-04-27T07:04:13.588397+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.588397+0000","last_undegraded":"2024-04-27T07:04:13.588397+0000","last_fullsized":"2024-04-27T07:04:13.588397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.583927+0000","last_change":"2024-04-27T07:04:13.583927+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.583927+0000","last_undegraded":"2024-04-27T07:04:13.583927+0000","last_fullsized":"2024-04-27T07:04:13.583927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.580842+0000","last_change":"2024-04-27T07:04:13.580842+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.580842+0000","last_undegraded":"2024-04-27T07:04:13.580842+0000","last_fullsized":"2024-04-27T07:04:13.580842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.586516+0000","last_change":"2024-04-27T07:04:13.586516+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.586516+0000","last_undegraded":"2024-04-27T07:04:13.586516+0000","last_fullsized":"2024-04-27T07:04:13.586516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.577547+0000","last_change":"2024-04-27T07:04:13.577547+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.577547+0000","last_undegraded":"2024-04-27T07:04:13.577547+0000","last_fullsized":"2024-04-27T07:04:13.577547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.584610+0000","last_change":"2024-04-27T07:04:13.584610+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.584610+0000","last_undegraded":"2024-04-27T07:04:13.584610+0000","last_fullsized":"2024-04-27T07:04:13.584610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.587198+0000","last_change":"2024-04-27T07:04:13.587198+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.587198+0000","last_undegraded":"2024-04-27T07:04:13.587198+0000","last_fullsized":"2024-04-27T07:04:13.587198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.582193+0000","last_change":"2024-04-27T07:04:13.582193+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.582193+0000","last_undegraded":"2024-04-27T07:04:13.582193+0000","last_fullsized":"2024-04-27T07:04:13.582193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573203+0000","last_change":"2024-04-27T07:04:13.573203+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573203+0000","last_undegraded":"2024-04-27T07:04:13.573203+0000","last_fullsized":"2024-04-27T07:04:13.573203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.582149+0000","last_change":"2024-04-27T07:04:13.582149+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.582149+0000","last_undegraded":"2024-04-27T07:04:13.582149+0000","last_fullsized":"2024-04-27T07:04:13.582149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579869+0000","last_change":"2024-04-27T07:04:13.579869+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579869+0000","last_undegraded":"2024-04-27T07:04:13.579869+0000","last_fullsized":"2024-04-27T07:04:13.579869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.583247+0000","last_change":"2024-04-27T07:04:13.583247+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.583247+0000","last_undegraded":"2024-04-27T07:04:13.583247+0000","last_fullsized":"2024-04-27T07:04:13.583247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-27T07:04:13.576472+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:04:13.576472+0000","last_peered":"2024-04-27T07:04:13.576472+0000","last_clean":"2024-04-27T07:04:13.576472+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:04:13.576472+0000","last_undegraded":"2024-04-27T07:04:13.576472+0000","last_fullsized":"2024-04-27T07:04:13.576472+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.578258+0000","last_change":"2024-04-27T07:04:13.578258+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.578258+0000","last_undegraded":"2024-04-27T07:04:13.578258+0000","last_fullsized":"2024-04-27T07:04:13.578258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573422+0000","last_change":"2024-04-27T07:04:13.573422+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573422+0000","last_undegraded":"2024-04-27T07:04:13.573422+0000","last_fullsized":"2024-04-27T07:04:13.573422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.582903+0000","last_change":"2024-04-27T07:04:13.582903+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.582903+0000","last_undegraded":"2024-04-27T07:04:13.582903+0000","last_fullsized":"2024-04-27T07:04:13.582903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579967+0000","last_change":"2024-04-27T07:04:13.579967+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579967+0000","last_undegraded":"2024-04-27T07:04:13.579967+0000","last_fullsized":"2024-04-27T07:04:13.579967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.579686+0000","last_change":"2024-04-27T07:04:13.579686+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.579686+0000","last_undegraded":"2024-04-27T07:04:13.579686+0000","last_fullsized":"2024-04-27T07:04:13.579686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573037+0000","last_change":"2024-04-27T07:04:13.573037+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573037+0000","last_undegraded":"2024-04-27T07:04:13.573037+0000","last_fullsized":"2024-04-27T07:04:13.573037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.581604+0000","last_change":"2024-04-27T07:04:13.581604+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.581604+0000","last_undegraded":"2024-04-27T07:04:13.581604+0000","last_fullsized":"2024-04-27T07:04:13.581604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.587674+0000","last_change":"2024-04-27T07:04:13.587674+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.587674+0000","last_undegraded":"2024-04-27T07:04:13.587674+0000","last_fullsized":"2024-04-27T07:04:13.587674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.574827+0000","last_change":"2024-04-27T07:04:13.574827+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.574827+0000","last_undegraded":"2024-04-27T07:04:13.574827+0000","last_fullsized":"2024-04-27T07:04:13.574827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.577590+0000","last_change":"2024-04-27T07:04:13.577590+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.577590+0000","last_undegraded":"2024-04-27T07:04:13.577590+0000","last_fullsized":"2024-04-27T07:04:13.577590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.578559+0000","last_change":"2024-04-27T07:04:13.578559+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.578559+0000","last_undegraded":"2024-04-27T07:04:13.578559+0000","last_fullsized":"2024-04-27T07:04:13.578559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573457+0000","last_change":"2024-04-27T07:04:13.573457+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573457+0000","last_undegraded":"2024-04-27T07:04:13.573457+0000","last_fullsized":"2024-04-27T07:04:13.573457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.573722+0000","last_change":"2024-04-27T07:04:13.573722+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.573722+0000","last_undegraded":"2024-04-27T07:04:13.573722+0000","last_fullsized":"2024-04-27T07:04:13.573722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.575544+0000","last_change":"2024-04-27T07:04:13.575544+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.575544+0000","last_undegraded":"2024-04-27T07:04:13.575544+0000","last_fullsized":"2024-04-27T07:04:13.575544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.576708+0000","last_change":"2024-04-27T07:04:13.576708+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.576708+0000","last_undegraded":"2024-04-27T07:04:13.576708+0000","last_fullsized":"2024-04-27T07:04:13.576708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-27T07:04:14.566367+0000","last_change":"2024-04-27T07:04:14.566367+0000","last_active":"2024-04-27T07:04:14.566367+0000","last_peered":"2024-04-27T07:04:14.566367+0000","last_clean":"2024-04-27T07:04:14.566367+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:14.566367+0000","last_undegraded":"2024-04-27T07:04:14.566367+0000","last_fullsized":"2024-04-27T07:04:14.566367+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-27T07:04:13.578313+0000","last_change":"2024-04-27T07:04:13.578313+0000","last_active":"2024-04-27T07:04:13.554072+0000","last_peered":"2024-04-27T07:04:13.554072+0000","last_clean":"2024-04-27T07:04:13.554072+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-27T07:04:13.578313+0000","last_undegraded":"2024-04-27T07:04:13.578313+0000","last_fullsized":"2024-04-27T07:04:13.578313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:04:20.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-04-27T07:04:20.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-04-27T07:04:20.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-04-27T07:04:20.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-04-27T07:04:20.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-04-27T07:04:20.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-04-27T07:04:20.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-04-27T07:04:20.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-04-27T07:04:20.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-04-27T07:04:20.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-04-27T07:04:20.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-04-27T07:04:20.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-04-27T07:04:20.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-04-27T07:04:20.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-04-27T07:04:20.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-04-27T07:04:20.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-04-27T07:04:20.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-04-27T07:04:20.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-04-27T07:04:20.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-04-27T07:04:20.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-04-27T07:04:20.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-04-27T07:04:20.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-04-27T07:04:20.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-04-27T07:04:20.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-04-27T07:04:20.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-04-27T07:04:20.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-04-27T07:04:20.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-04-27T07:04:20.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-04-27T07:04:20.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-04-27T07:04:20.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.576639+0000', 'last_change': '2024-04-27T07:04:13.576639+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.576639+0000', 'last_undegraded': '2024-04-27T07:04:13.576639+0000', 'last_fullsized': '2024-04-27T07:04:13.576639+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-04-27T07:04:20.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.585621+0000', 'last_change': '2024-04-27T07:04:13.585621+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.585621+0000', 'last_undegraded': '2024-04-27T07:04:13.585621+0000', 'last_fullsized': '2024-04-27T07:04:13.585621+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.577307+0000', 'last_change': '2024-04-27T07:04:13.577307+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.577307+0000', 'last_undegraded': '2024-04-27T07:04:13.577307+0000', 'last_fullsized': '2024-04-27T07:04:13.577307+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.579242+0000', 'last_change': '2024-04-27T07:04:13.579242+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.579242+0000', 'last_undegraded': '2024-04-27T07:04:13.579242+0000', 'last_fullsized': '2024-04-27T07:04:13.579242+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-04-27T07:04:20.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.588397+0000', 'last_change': '2024-04-27T07:04:13.588397+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.588397+0000', 'last_undegraded': '2024-04-27T07:04:13.588397+0000', 'last_fullsized': '2024-04-27T07:04:13.588397+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.583927+0000', 'last_change': '2024-04-27T07:04:13.583927+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.583927+0000', 'last_undegraded': '2024-04-27T07:04:13.583927+0000', 'last_fullsized': '2024-04-27T07:04:13.583927+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.580842+0000', 'last_change': '2024-04-27T07:04:13.580842+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.580842+0000', 'last_undegraded': '2024-04-27T07:04:13.580842+0000', 'last_fullsized': '2024-04-27T07:04:13.580842+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-04-27T07:04:20.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.586516+0000', 'last_change': '2024-04-27T07:04:13.586516+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.586516+0000', 'last_undegraded': '2024-04-27T07:04:13.586516+0000', 'last_fullsized': '2024-04-27T07:04:13.586516+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.577547+0000', 'last_change': '2024-04-27T07:04:13.577547+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.577547+0000', 'last_undegraded': '2024-04-27T07:04:13.577547+0000', 'last_fullsized': '2024-04-27T07:04:13.577547+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.584610+0000', 'last_change': '2024-04-27T07:04:13.584610+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.584610+0000', 'last_undegraded': '2024-04-27T07:04:13.584610+0000', 'last_fullsized': '2024-04-27T07:04:13.584610+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-04-27T07:04:20.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.587198+0000', 'last_change': '2024-04-27T07:04:13.587198+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.587198+0000', 'last_undegraded': '2024-04-27T07:04:13.587198+0000', 'last_fullsized': '2024-04-27T07:04:13.587198+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.582193+0000', 'last_change': '2024-04-27T07:04:13.582193+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.582193+0000', 'last_undegraded': '2024-04-27T07:04:13.582193+0000', 'last_fullsized': '2024-04-27T07:04:13.582193+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.573203+0000', 'last_change': '2024-04-27T07:04:13.573203+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.573203+0000', 'last_undegraded': '2024-04-27T07:04:13.573203+0000', 'last_fullsized': '2024-04-27T07:04:13.573203+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-04-27T07:04:20.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.582149+0000', 'last_change': '2024-04-27T07:04:13.582149+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.582149+0000', 'last_undegraded': '2024-04-27T07:04:13.582149+0000', 'last_fullsized': '2024-04-27T07:04:13.582149+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.579869+0000', 'last_change': '2024-04-27T07:04:13.579869+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.579869+0000', 'last_undegraded': '2024-04-27T07:04:13.579869+0000', 'last_fullsized': '2024-04-27T07:04:13.579869+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.583247+0000', 'last_change': '2024-04-27T07:04:13.583247+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.583247+0000', 'last_undegraded': '2024-04-27T07:04:13.583247+0000', 'last_fullsized': '2024-04-27T07:04:13.583247+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.578258+0000', 'last_change': '2024-04-27T07:04:13.578258+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.578258+0000', 'last_undegraded': '2024-04-27T07:04:13.578258+0000', 'last_fullsized': '2024-04-27T07:04:13.578258+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.573422+0000', 'last_change': '2024-04-27T07:04:13.573422+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.573422+0000', 'last_undegraded': '2024-04-27T07:04:13.573422+0000', 'last_fullsized': '2024-04-27T07:04:13.573422+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.582903+0000', 'last_change': '2024-04-27T07:04:13.582903+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.582903+0000', 'last_undegraded': '2024-04-27T07:04:13.582903+0000', 'last_fullsized': '2024-04-27T07:04:13.582903+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-04-27T07:04:20.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.579967+0000', 'last_change': '2024-04-27T07:04:13.579967+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.579967+0000', 'last_undegraded': '2024-04-27T07:04:13.579967+0000', 'last_fullsized': '2024-04-27T07:04:13.579967+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.579686+0000', 'last_change': '2024-04-27T07:04:13.579686+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.579686+0000', 'last_undegraded': '2024-04-27T07:04:13.579686+0000', 'last_fullsized': '2024-04-27T07:04:13.579686+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.573037+0000', 'last_change': '2024-04-27T07:04:13.573037+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.573037+0000', 'last_undegraded': '2024-04-27T07:04:13.573037+0000', 'last_fullsized': '2024-04-27T07:04:13.573037+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.581604+0000', 'last_change': '2024-04-27T07:04:13.581604+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.581604+0000', 'last_undegraded': '2024-04-27T07:04:13.581604+0000', 'last_fullsized': '2024-04-27T07:04:13.581604+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:04:20.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.587674+0000', 'last_change': '2024-04-27T07:04:13.587674+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.587674+0000', 'last_undegraded': '2024-04-27T07:04:13.587674+0000', 'last_fullsized': '2024-04-27T07:04:13.587674+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.574827+0000', 'last_change': '2024-04-27T07:04:13.574827+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.574827+0000', 'last_undegraded': '2024-04-27T07:04:13.574827+0000', 'last_fullsized': '2024-04-27T07:04:13.574827+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.577590+0000', 'last_change': '2024-04-27T07:04:13.577590+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.577590+0000', 'last_undegraded': '2024-04-27T07:04:13.577590+0000', 'last_fullsized': '2024-04-27T07:04:13.577590+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-04-27T07:04:20.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-04-27T07:04:20.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.578559+0000', 'last_change': '2024-04-27T07:04:13.578559+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.578559+0000', 'last_undegraded': '2024-04-27T07:04:13.578559+0000', 'last_fullsized': '2024-04-27T07:04:13.578559+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-27T07:04:20.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-04-27T07:04:20.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-04-27T07:04:20.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.573457+0000', 'last_change': '2024-04-27T07:04:13.573457+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.573457+0000', 'last_undegraded': '2024-04-27T07:04:13.573457+0000', 'last_fullsized': '2024-04-27T07:04:13.573457+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:04:20.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-04-27T07:04:20.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-04-27T07:04:20.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-04-27T07:04:20.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.573722+0000', 'last_change': '2024-04-27T07:04:13.573722+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.573722+0000', 'last_undegraded': '2024-04-27T07:04:13.573722+0000', 'last_fullsized': '2024-04-27T07:04:13.573722+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:04:20.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-04-27T07:04:20.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-04-27T07:04:20.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.575544+0000', 'last_change': '2024-04-27T07:04:13.575544+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.575544+0000', 'last_undegraded': '2024-04-27T07:04:13.575544+0000', 'last_fullsized': '2024-04-27T07:04:13.575544+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-27T07:04:20.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-04-27T07:04:20.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-04-27T07:04:20.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.576708+0000', 'last_change': '2024-04-27T07:04:13.576708+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.576708+0000', 'last_undegraded': '2024-04-27T07:04:13.576708+0000', 'last_fullsized': '2024-04-27T07:04:13.576708+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-27T07:04:20.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-04-27T07:04:20.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-27T07:04:14.566367+0000', 'last_change': '2024-04-27T07:04:14.566367+0000', 'last_active': '2024-04-27T07:04:14.566367+0000', 'last_peered': '2024-04-27T07:04:14.566367+0000', 'last_clean': '2024-04-27T07:04:14.566367+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:14.566367+0000', 'last_undegraded': '2024-04-27T07:04:14.566367+0000', 'last_fullsized': '2024-04-27T07:04:14.566367+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:14.566367+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-27T07:04:20.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-04-27T07:04:20.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2024-04-27T07:04:13.578313+0000', 'last_change': '2024-04-27T07:04:13.578313+0000', 'last_active': '2024-04-27T07:04:13.554072+0000', 'last_peered': '2024-04-27T07:04:13.554072+0000', 'last_clean': '2024-04-27T07:04:13.554072+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-27T07:04:13.578313+0000', 'last_undegraded': '2024-04-27T07:04:13.578313+0000', 'last_fullsized': '2024-04-27T07:04:13.578313+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'last_clean_scrub_stamp': '2024-04-27T07:04:13.554072+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-27T07:04:20.424 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-27T07:04:20.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.557+0000 7f906c4b8640 1 Processor -- start 2024-04-27T07:04:20.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.557+0000 7f906c4b8640 1 -- start start 2024-04-27T07:04:20.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.561+0000 7f906c4b8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90640b9a10 0x7f90640739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.561+0000 7f906c4b8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90640b9df0 con 0x7f9064073ee0 2024-04-27T07:04:20.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.561+0000 7f906c4b8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90640b47e0 con 0x7f906406ddc0 2024-04-27T07:04:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.561+0000 7f906c4b8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f90640b4590 con 0x7f90640b9a10 2024-04-27T07:04:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.561+0000 7f906a254640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90640b9a10 0x7f90640739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.561+0000 7f906aa55640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9064073ee0 0x7f90641525b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40806/0 (socket says 172.21.15.12:40806) 2024-04-27T07:04:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.561+0000 7f906aa55640 1 -- 172.21.15.12:0/941397486 learned_addr learned my addr 172.21.15.12:0/941397486 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.561+0000 7f906a254640 1 -- 172.21.15.12:0/941397486 >> v1:172.21.15.53:6789/0 conn(0x7f906406ddc0 legacy=0x7f9064148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.561+0000 7f906a254640 1 -- 172.21.15.12:0/941397486 >> v1:172.21.15.12:6789/0 conn(0x7f9064073ee0 legacy=0x7f90641525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.561+0000 7f906a254640 1 -- 172.21.15.12:0/941397486 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90640b4ac0 con 0x7f90640b9a10 2024-04-27T07:04:20.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906a254640 1 --2- 172.21.15.12:0/941397486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 0x7f90640739a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f90540097c0 tx=0x7f905401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb008b8875928fad server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:20.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069252640 1 -- 172.21.15.12:0/941397486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90540b0070 con 0x7f90640b9a10 2024-04-27T07:04:20.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069252640 1 -- 172.21.15.12:0/941397486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90540ab600 con 0x7f90640b9a10 2024-04-27T07:04:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069252640 1 -- 172.21.15.12:0/941397486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90540ab9d0 con 0x7f90640b9a10 2024-04-27T07:04:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 -- 172.21.15.12:0/941397486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 msgr2=0x7f90640739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 --2- 172.21.15.12:0/941397486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 0x7f90640739a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f90540097c0 tx=0x7f905401c340 comp rx=0 tx=0).stop 2024-04-27T07:04:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 -- 172.21.15.12:0/941397486 shutdown_connections 2024-04-27T07:04:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 --2- 172.21.15.12:0/941397486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 0x7f90640739a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 -- 172.21.15.12:0/941397486 >> 172.21.15.12:0/941397486 conn(0x7f9064067d10 msgr2=0x7f9064068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 -- 172.21.15.12:0/941397486 shutdown_connections 2024-04-27T07:04:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 -- 172.21.15.12:0/941397486 wait complete. 2024-04-27T07:04:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 Processor -- start 2024-04-27T07:04:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 -- start start 2024-04-27T07:04:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9064073ee0 0x7f906413c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 0x7f90640b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90641e0e90 0x7f90641e3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f906406db60 con 0x7f90640b9a10 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f906406d9e0 con 0x7f90641e0e90 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906c4b8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f90641564f0 con 0x7f9064073ee0 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069a53640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 0x7f90640b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069a53640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 0x7f90640b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41980/0 (socket says 172.21.15.12:41980) 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069a53640 1 -- 172.21.15.12:0/2505192277 learned_addr learned my addr 172.21.15.12:0/2505192277 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906aa55640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90641e0e90 0x7f90641e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906a254640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9064073ee0 0x7f906413c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069a53640 1 -- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9064073ee0 msgr2=0x7f906413c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069a53640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9064073ee0 0x7f906413c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069a53640 1 -- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90641e0e90 msgr2=0x7f90641e3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069a53640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90641e0e90 0x7f90641e3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069a53640 1 -- 172.21.15.12:0/2505192277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90641e03d0 con 0x7f90640b9a10 2024-04-27T07:04:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906aa55640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90641e0e90 0x7f90641e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f906a254640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9064073ee0 0x7f906413c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f9069a53640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 0x7f90640b1fa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f906000c480 tx=0x7f906000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.565+0000 7f905affd640 1 -- 172.21.15.12:0/2505192277 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060012830 con 0x7f90640b9a10 2024-04-27T07:04:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.569+0000 7f905affd640 1 -- 172.21.15.12:0/2505192277 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9060017070 con 0x7f90640b9a10 2024-04-27T07:04:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.569+0000 7f905affd640 1 -- 172.21.15.12:0/2505192277 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060012c30 con 0x7f90640b9a10 2024-04-27T07:04:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.569+0000 7f906c4b8640 1 -- 172.21.15.12:0/2505192277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90641e0690 con 0x7f90640b9a10 2024-04-27T07:04:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.569+0000 7f906c4b8640 1 -- 172.21.15.12:0/2505192277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90640b0790 con 0x7f90640b9a10 2024-04-27T07:04:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.569+0000 7f906c4b8640 1 -- 172.21.15.12:0/2505192277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9064068c20 con 0x7f90640b9a10 2024-04-27T07:04:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.569+0000 7f905affd640 1 -- 172.21.15.12:0/2505192277 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f906001c460 con 0x7f90640b9a10 2024-04-27T07:04:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.569+0000 7f905affd640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9048068380 0x7f904806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.569+0000 7f905affd640 1 -- 172.21.15.12:0/2505192277 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9060086700 con 0x7f90640b9a10 2024-04-27T07:04:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.569+0000 7f906a254640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9048068380 0x7f904806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.569+0000 7f906a254640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9048068380 0x7f904806a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f90540ad000 tx=0x7f9054008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:20.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.573+0000 7f905affd640 1 -- 172.21.15.12:0/2505192277 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9060056d50 con 0x7f90640b9a10 2024-04-27T07:04:20.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.793+0000 7f906c4b8640 1 -- 172.21.15.12:0/2505192277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f90641e12a0 con 0x7f90640b9a10 2024-04-27T07:04:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f905affd640 1 -- 172.21.15.12:0/2505192277 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1011 (secure 0 0 0) 0x7f9060056650 con 0x7f90640b9a10 2024-04-27T07:04:20.800 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:20.800 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":23,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1714201448,"num_in_osds":8,"osd_in_since":1714201440,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":241291264,"bytes_avail":772852822016,"bytes_total":773094113280,"write_bytes_sec":2377,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4242}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-27T07:03:57.717068+0000","services":{}},"progress_events":{}} 2024-04-27T07:04:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 -- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9048068380 msgr2=0x7f904806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9048068380 0x7f904806a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f90540ad000 tx=0x7f9054008040 comp rx=0 tx=0).stop 2024-04-27T07:04:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 -- 172.21.15.12:0/2505192277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 msgr2=0x7f90640b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 0x7f90640b1fa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f906000c480 tx=0x7f906000c940 comp rx=0 tx=0).stop 2024-04-27T07:04:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 -- 172.21.15.12:0/2505192277 shutdown_connections 2024-04-27T07:04:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9048068380 0x7f904806a840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90641e0e90 0x7f90641e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90640b9a10 0x7f90640b1fa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 --2- 172.21.15.12:0/2505192277 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9064073ee0 0x7f906413c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 -- 172.21.15.12:0/2505192277 >> 172.21.15.12:0/2505192277 conn(0x7f9064067d10 msgr2=0x7f9064147500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:20.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 -- 172.21.15.12:0/2505192277 shutdown_connections 2024-04-27T07:04:20.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.797+0000 7f906c4b8640 1 -- 172.21.15.12:0/2505192277 wait complete. 2024-04-27T07:04:20.817 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-27T07:04:20.818 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:04:20.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.933+0000 7ff7f8a82640 1 Processor -- start 2024-04-27T07:04:20.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.933+0000 7ff7f8a82640 1 -- start start 2024-04-27T07:04:20.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f8a82640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7ff7f40b9a10 0x7ff7f4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f8a82640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f40b9df0 con 0x7ff7f4144fa0 2024-04-27T07:04:20.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f8a82640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f409e0e0 con 0x7ff7f409d980 2024-04-27T07:04:20.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f8a82640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f4144b10 con 0x7ff7f40b9a10 2024-04-27T07:04:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f2d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff7f4144fa0 0x7ff7f41453f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40812/0 (socket says 172.21.15.12:40812) 2024-04-27T07:04:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f2d9d640 1 -- 172.21.15.12:0/363348045 learned_addr learned my addr 172.21.15.12:0/363348045 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f259c640 1 --2- 172.21.15.12:0/363348045 >> v2:172.21.15.53:3300/0 conn(0x7ff7f40b9a10 0x7ff7f4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1628431481 0 0) 0x7ff7f40b9df0 con 0x7ff7f4144fa0 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7dc003530 con 0x7ff7f4144fa0 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1362312936 0 0) 0x7ff7dc003530 con 0x7ff7f4144fa0 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 >> v2:172.21.15.53:3300/0 conn(0x7ff7f40b9a10 msgr2=0x7ff7f4073c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 --2- 172.21.15.12:0/363348045 >> v2:172.21.15.53:3300/0 conn(0x7ff7f40b9a10 0x7ff7f4073c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 >> v1:172.21.15.53:6789/0 conn(0x7ff7f409d980 legacy=0x7ff7f413e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f409e3c0 con 0x7ff7f4144fa0 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7ff7ec001a00 con 0x7ff7f4144fa0 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 >> v1:172.21.15.12:6789/0 conn(0x7ff7f4144fa0 legacy=0x7ff7f41453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7dc0039c0 0x7ff7dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff7dc004810 0x7ff7dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4144b10 con 0x7ff7dc0051d0 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc00c080 con 0x7ff7dc0039c0 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7ff7dc00c280 con 0x7ff7dc004810 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f2d9d640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f1d9b640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff7dc004810 0x7ff7dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f259c640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7dc0039c0 0x7ff7dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f2d9d640 1 -- 172.21.15.12:0/363348045 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff7dc004810 msgr2=0x7ff7dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f2d9d640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff7dc004810 0x7ff7dc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f2d9d640 1 -- 172.21.15.12:0/363348045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7dc0039c0 msgr2=0x7ff7dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f2d9d640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7dc0039c0 0x7ff7dc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.937+0000 7ff7f2d9d640 1 -- 172.21.15.12:0/363348045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc003530 con 0x7ff7dc0051d0 2024-04-27T07:04:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f2d9d640 1 -- 172.21.15.12:0/363348045 reap_dead start 2024-04-27T07:04:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f2d9d640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7dc00b9d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00a840 tx=0x7ff7ec01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcd517d620f884c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00a460 con 0x7ff7dc0051d0 2024-04-27T07:04:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec0ac640 con 0x7ff7dc0051d0 2024-04-27T07:04:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f159a640 1 -- 172.21.15.12:0/363348045 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0aca30 con 0x7ff7dc0051d0 2024-04-27T07:04:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 -- 172.21.15.12:0/363348045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 msgr2=0x7ff7dc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7dc00b9d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00a840 tx=0x7ff7ec01d360 comp rx=0 tx=0).stop 2024-04-27T07:04:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 -- 172.21.15.12:0/363348045 shutdown_connections 2024-04-27T07:04:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 --2- 172.21.15.12:0/363348045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7dc00b9d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 -- 172.21.15.12:0/363348045 >> 172.21.15.12:0/363348045 conn(0x7ff7f4067d10 msgr2=0x7ff7f4068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 -- 172.21.15.12:0/363348045 shutdown_connections 2024-04-27T07:04:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 -- 172.21.15.12:0/363348045 wait complete. 2024-04-27T07:04:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 Processor -- start 2024-04-27T07:04:20.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 -- start start 2024-04-27T07:04:20.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7f4072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff7f41deab0 0x7ff7f413d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7f4072f70 0x7ff7f413d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4073de0 con 0x7ff7dc0051d0 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4073c60 con 0x7ff7f4072f70 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f4073f60 con 0x7ff7f41deab0 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7f4072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7f4072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41988/0 (socket says 172.21.15.12:41988) 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f259c640 1 -- 172.21.15.12:0/118341845 learned_addr learned my addr 172.21.15.12:0/118341845 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f2d9d640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7f4072f70 0x7ff7f413d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f1d9b640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff7f41deab0 0x7ff7f413d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f259c640 1 -- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff7f41deab0 msgr2=0x7ff7f413d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f259c640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff7f41deab0 0x7ff7f413d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f259c640 1 -- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7f4072f70 msgr2=0x7ff7f413d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f259c640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7f4072f70 0x7ff7f413d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:20.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f259c640 1 -- 172.21.15.12:0/118341845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f41def70 con 0x7ff7dc0051d0 2024-04-27T07:04:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f259c640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7f4072230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4011560 tx=0x7ff7e4011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7e2ffd640 1 -- 172.21.15.12:0/118341845 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4015930 con 0x7ff7dc0051d0 2024-04-27T07:04:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7e2ffd640 1 -- 172.21.15.12:0/118341845 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e4015ad0 con 0x7ff7dc0051d0 2024-04-27T07:04:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7f8a82640 1 -- 172.21.15.12:0/118341845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f41df250 con 0x7ff7dc0051d0 2024-04-27T07:04:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.941+0000 7ff7e2ffd640 1 -- 172.21.15.12:0/118341845 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400f460 con 0x7ff7dc0051d0 2024-04-27T07:04:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.945+0000 7ff7f8a82640 1 -- 172.21.15.12:0/118341845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f40b8bd0 con 0x7ff7dc0051d0 2024-04-27T07:04:20.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.945+0000 7ff7f8a82640 1 -- 172.21.15.12:0/118341845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f4068c20 con 0x7ff7dc0051d0 2024-04-27T07:04:20.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.945+0000 7ff7e2ffd640 1 -- 172.21.15.12:0/118341845 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff7e400f600 con 0x7ff7dc0051d0 2024-04-27T07:04:20.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.945+0000 7ff7e2ffd640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff7d40683b0 0x7ff7d406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:20.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.945+0000 7ff7e2ffd640 1 -- 172.21.15.12:0/118341845 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff7e4087250 con 0x7ff7dc0051d0 2024-04-27T07:04:20.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.945+0000 7ff7f1d9b640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff7d40683b0 0x7ff7d406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:20.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.945+0000 7ff7f1d9b640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff7d40683b0 0x7ff7d406a870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4071b10 tx=0x7ff7e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:20.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:20.949+0000 7ff7e2ffd640 1 -- 172.21.15.12:0/118341845 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff7e4019030 con 0x7ff7dc0051d0 2024-04-27T07:04:21.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.105+0000 7ff7f8a82640 1 -- 172.21.15.12:0/118341845 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff7f40b4cf0 con 0x7ff7d40683b0 2024-04-27T07:04:21.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.125+0000 7ff7e2ffd640 1 -- 172.21.15.12:0/118341845 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7ff7f40b4cf0 con 0x7ff7d40683b0 2024-04-27T07:04:21.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:21.134 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 -- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff7d40683b0 msgr2=0x7ff7d406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff7d40683b0 0x7ff7d406a870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4071b10 tx=0x7ff7e8009040 comp rx=0 tx=0).stop 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 -- 172.21.15.12:0/118341845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 msgr2=0x7ff7f4072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7f4072230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4011560 tx=0x7ff7e4011a20 comp rx=0 tx=0).stop 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 -- 172.21.15.12:0/118341845 shutdown_connections 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff7d40683b0 0x7ff7d406a870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7f4072f70 0x7ff7f413d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff7f41deab0 0x7ff7f413d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 --2- 172.21.15.12:0/118341845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7dc0051d0 0x7ff7f4072230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 -- 172.21.15.12:0/118341845 >> 172.21.15.12:0/118341845 conn(0x7ff7f4067d10 msgr2=0x7ff7f413e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 -- 172.21.15.12:0/118341845 shutdown_connections 2024-04-27T07:04:21.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:21.129+0000 7ff7f8a82640 1 -- 172.21.15.12:0/118341845 wait complete. 2024-04-27T07:04:21.151 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-04-27T07:04:20.609632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235636,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739084,"statfs":{"total":773094113280,"available":772852822016,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.030837"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495786+0000","last_change":"2024-04-27T07:04:15.652893+0000","last_active":"2024-04-27T07:04:17.495786+0000","last_peered":"2024-04-27T07:04:17.495786+0000","last_clean":"2024-04-27T07:04:17.495786+0000","last_became_active":"2024-04-27T07:04:15.652155+0000","last_became_peered":"2024-04-27T07:04:15.652155+0000","last_unstale":"2024-04-27T07:04:17.495786+0000","last_undegraded":"2024-04-27T07:04:17.495786+0000","last_fullsized":"2024-04-27T07:04:17.495786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:07:07.313585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610741+0000","last_change":"2024-04-27T07:04:15.622035+0000","last_active":"2024-04-27T07:04:16.610741+0000","last_peered":"2024-04-27T07:04:16.610741+0000","last_clean":"2024-04-27T07:04:16.610741+0000","last_became_active":"2024-04-27T07:04:15.621365+0000","last_became_peered":"2024-04-27T07:04:15.621365+0000","last_unstale":"2024-04-27T07:04:16.610741+0000","last_undegraded":"2024-04-27T07:04:16.610741+0000","last_fullsized":"2024-04-27T07:04:16.610741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494175+0000","last_change":"2024-04-27T07:04:15.603108+0000","last_active":"2024-04-27T07:04:17.494175+0000","last_peered":"2024-04-27T07:04:17.494175+0000","last_clean":"2024-04-27T07:04:17.494175+0000","last_became_active":"2024-04-27T07:04:15.602672+0000","last_became_peered":"2024-04-27T07:04:15.602672+0000","last_unstale":"2024-04-27T07:04:17.494175+0000","last_undegraded":"2024-04-27T07:04:17.494175+0000","last_fullsized":"2024-04-27T07:04:17.494175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494169+0000","last_change":"2024-04-27T07:04:15.621096+0000","last_active":"2024-04-27T07:04:17.494169+0000","last_peered":"2024-04-27T07:04:17.494169+0000","last_clean":"2024-04-27T07:04:17.494169+0000","last_became_active":"2024-04-27T07:04:15.618832+0000","last_became_peered":"2024-04-27T07:04:15.618832+0000","last_unstale":"2024-04-27T07:04:17.494169+0000","last_undegraded":"2024-04-27T07:04:17.494169+0000","last_fullsized":"2024-04-27T07:04:17.494169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:08.890952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494305+0000","last_change":"2024-04-27T07:04:15.627875+0000","last_active":"2024-04-27T07:04:17.494305+0000","last_peered":"2024-04-27T07:04:17.494305+0000","last_clean":"2024-04-27T07:04:17.494305+0000","last_became_active":"2024-04-27T07:04:15.626904+0000","last_became_peered":"2024-04-27T07:04:15.626904+0000","last_unstale":"2024-04-27T07:04:17.494305+0000","last_undegraded":"2024-04-27T07:04:17.494305+0000","last_fullsized":"2024-04-27T07:04:17.494305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:11.386203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494240+0000","last_change":"2024-04-27T07:04:15.632294+0000","last_active":"2024-04-27T07:04:17.494240+0000","last_peered":"2024-04-27T07:04:17.494240+0000","last_clean":"2024-04-27T07:04:17.494240+0000","last_became_active":"2024-04-27T07:04:15.630191+0000","last_became_peered":"2024-04-27T07:04:15.630191+0000","last_unstale":"2024-04-27T07:04:17.494240+0000","last_undegraded":"2024-04-27T07:04:17.494240+0000","last_fullsized":"2024-04-27T07:04:17.494240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:19.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493321+0000","last_change":"2024-04-27T07:04:15.652899+0000","last_active":"2024-04-27T07:04:17.493321+0000","last_peered":"2024-04-27T07:04:17.493321+0000","last_clean":"2024-04-27T07:04:17.493321+0000","last_became_active":"2024-04-27T07:04:15.652383+0000","last_became_peered":"2024-04-27T07:04:15.652383+0000","last_unstale":"2024-04-27T07:04:17.493321+0000","last_undegraded":"2024-04-27T07:04:17.493321+0000","last_fullsized":"2024-04-27T07:04:17.493321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495664+0000","last_change":"2024-04-27T07:04:15.635192+0000","last_active":"2024-04-27T07:04:17.495664+0000","last_peered":"2024-04-27T07:04:17.495664+0000","last_clean":"2024-04-27T07:04:17.495664+0000","last_became_active":"2024-04-27T07:04:15.634845+0000","last_became_peered":"2024-04-27T07:04:15.634845+0000","last_unstale":"2024-04-27T07:04:17.495664+0000","last_undegraded":"2024-04-27T07:04:17.495664+0000","last_fullsized":"2024-04-27T07:04:17.495664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495704+0000","last_change":"2024-04-27T07:04:15.642203+0000","last_active":"2024-04-27T07:04:17.495704+0000","last_peered":"2024-04-27T07:04:17.495704+0000","last_clean":"2024-04-27T07:04:17.495704+0000","last_became_active":"2024-04-27T07:04:15.640450+0000","last_became_peered":"2024-04-27T07:04:15.640450+0000","last_unstale":"2024-04-27T07:04:17.495704+0000","last_undegraded":"2024-04-27T07:04:17.495704+0000","last_fullsized":"2024-04-27T07:04:17.495704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:46.491888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614056+0000","last_change":"2024-04-27T07:04:15.620338+0000","last_active":"2024-04-27T07:04:16.614056+0000","last_peered":"2024-04-27T07:04:16.614056+0000","last_clean":"2024-04-27T07:04:16.614056+0000","last_became_active":"2024-04-27T07:04:15.619521+0000","last_became_peered":"2024-04-27T07:04:15.619521+0000","last_unstale":"2024-04-27T07:04:16.614056+0000","last_undegraded":"2024-04-27T07:04:16.614056+0000","last_fullsized":"2024-04-27T07:04:16.614056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613051+0000","last_change":"2024-04-27T07:04:15.625615+0000","last_active":"2024-04-27T07:04:16.613051+0000","last_peered":"2024-04-27T07:04:16.613051+0000","last_clean":"2024-04-27T07:04:16.613051+0000","last_became_active":"2024-04-27T07:04:15.624734+0000","last_became_peered":"2024-04-27T07:04:15.624734+0000","last_unstale":"2024-04-27T07:04:16.613051+0000","last_undegraded":"2024-04-27T07:04:16.613051+0000","last_fullsized":"2024-04-27T07:04:16.613051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613805+0000","last_change":"2024-04-27T07:04:15.609501+0000","last_active":"2024-04-27T07:04:16.613805+0000","last_peered":"2024-04-27T07:04:16.613805+0000","last_clean":"2024-04-27T07:04:16.613805+0000","last_became_active":"2024-04-27T07:04:15.603709+0000","last_became_peered":"2024-04-27T07:04:15.603709+0000","last_unstale":"2024-04-27T07:04:16.613805+0000","last_undegraded":"2024-04-27T07:04:16.613805+0000","last_fullsized":"2024-04-27T07:04:16.613805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:59.982527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496346+0000","last_change":"2024-04-27T07:04:15.625180+0000","last_active":"2024-04-27T07:04:17.496346+0000","last_peered":"2024-04-27T07:04:17.496346+0000","last_clean":"2024-04-27T07:04:17.496346+0000","last_became_active":"2024-04-27T07:04:15.624748+0000","last_became_peered":"2024-04-27T07:04:15.624748+0000","last_unstale":"2024-04-27T07:04:17.496346+0000","last_undegraded":"2024-04-27T07:04:17.496346+0000","last_fullsized":"2024-04-27T07:04:17.496346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493065+0000","last_change":"2024-04-27T07:04:15.604939+0000","last_active":"2024-04-27T07:04:17.493065+0000","last_peered":"2024-04-27T07:04:17.493065+0000","last_clean":"2024-04-27T07:04:17.493065+0000","last_became_active":"2024-04-27T07:04:15.603580+0000","last_became_peered":"2024-04-27T07:04:15.603580+0000","last_unstale":"2024-04-27T07:04:17.493065+0000","last_undegraded":"2024-04-27T07:04:17.493065+0000","last_fullsized":"2024-04-27T07:04:17.493065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:45.914758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610468+0000","last_change":"2024-04-27T07:04:15.629060+0000","last_active":"2024-04-27T07:04:16.610468+0000","last_peered":"2024-04-27T07:04:16.610468+0000","last_clean":"2024-04-27T07:04:16.610468+0000","last_became_active":"2024-04-27T07:04:15.628737+0000","last_became_peered":"2024-04-27T07:04:15.628737+0000","last_unstale":"2024-04-27T07:04:16.610468+0000","last_undegraded":"2024-04-27T07:04:16.610468+0000","last_fullsized":"2024-04-27T07:04:16.610468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492723+0000","last_change":"2024-04-27T07:04:15.610164+0000","last_active":"2024-04-27T07:04:17.492723+0000","last_peered":"2024-04-27T07:04:17.492723+0000","last_clean":"2024-04-27T07:04:17.492723+0000","last_became_active":"2024-04-27T07:04:15.609290+0000","last_became_peered":"2024-04-27T07:04:15.609290+0000","last_unstale":"2024-04-27T07:04:17.492723+0000","last_undegraded":"2024-04-27T07:04:17.492723+0000","last_fullsized":"2024-04-27T07:04:17.492723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494381+0000","last_change":"2024-04-27T07:04:15.647816+0000","last_active":"2024-04-27T07:04:17.494381+0000","last_peered":"2024-04-27T07:04:17.494381+0000","last_clean":"2024-04-27T07:04:17.494381+0000","last_became_active":"2024-04-27T07:04:15.646892+0000","last_became_peered":"2024-04-27T07:04:15.646892+0000","last_unstale":"2024-04-27T07:04:17.494381+0000","last_undegraded":"2024-04-27T07:04:17.494381+0000","last_fullsized":"2024-04-27T07:04:17.494381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:01.732848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493299+0000","last_change":"2024-04-27T07:04:15.609407+0000","last_active":"2024-04-27T07:04:17.493299+0000","last_peered":"2024-04-27T07:04:17.493299+0000","last_clean":"2024-04-27T07:04:17.493299+0000","last_became_active":"2024-04-27T07:04:15.608565+0000","last_became_peered":"2024-04-27T07:04:15.608565+0000","last_unstale":"2024-04-27T07:04:17.493299+0000","last_undegraded":"2024-04-27T07:04:17.493299+0000","last_fullsized":"2024-04-27T07:04:17.493299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:37.376293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495719+0000","last_change":"2024-04-27T07:04:15.627082+0000","last_active":"2024-04-27T07:04:17.495719+0000","last_peered":"2024-04-27T07:04:17.495719+0000","last_clean":"2024-04-27T07:04:17.495719+0000","last_became_active":"2024-04-27T07:04:15.626302+0000","last_became_peered":"2024-04-27T07:04:15.626302+0000","last_unstale":"2024-04-27T07:04:17.495719+0000","last_undegraded":"2024-04-27T07:04:17.495719+0000","last_fullsized":"2024-04-27T07:04:17.495719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:01.732848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615116+0000","last_change":"2024-04-27T07:04:15.610661+0000","last_active":"2024-04-27T07:04:16.615116+0000","last_peered":"2024-04-27T07:04:16.615116+0000","last_clean":"2024-04-27T07:04:16.615116+0000","last_became_active":"2024-04-27T07:04:15.609899+0000","last_became_peered":"2024-04-27T07:04:15.609899+0000","last_unstale":"2024-04-27T07:04:16.615116+0000","last_undegraded":"2024-04-27T07:04:16.615116+0000","last_fullsized":"2024-04-27T07:04:16.615116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494321+0000","last_change":"2024-04-27T07:04:15.599714+0000","last_active":"2024-04-27T07:04:17.494321+0000","last_peered":"2024-04-27T07:04:17.494321+0000","last_clean":"2024-04-27T07:04:17.494321+0000","last_became_active":"2024-04-27T07:04:15.598855+0000","last_became_peered":"2024-04-27T07:04:15.598855+0000","last_unstale":"2024-04-27T07:04:17.494321+0000","last_undegraded":"2024-04-27T07:04:17.494321+0000","last_fullsized":"2024-04-27T07:04:17.494321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493334+0000","last_change":"2024-04-27T07:04:15.634881+0000","last_active":"2024-04-27T07:04:17.493334+0000","last_peered":"2024-04-27T07:04:17.493334+0000","last_clean":"2024-04-27T07:04:17.493334+0000","last_became_active":"2024-04-27T07:04:15.629428+0000","last_became_peered":"2024-04-27T07:04:15.629428+0000","last_unstale":"2024-04-27T07:04:17.493334+0000","last_undegraded":"2024-04-27T07:04:17.493334+0000","last_fullsized":"2024-04-27T07:04:17.493334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:40:20.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495828+0000","last_change":"2024-04-27T07:04:15.630015+0000","last_active":"2024-04-27T07:04:17.495828+0000","last_peered":"2024-04-27T07:04:17.495828+0000","last_clean":"2024-04-27T07:04:17.495828+0000","last_became_active":"2024-04-27T07:04:15.629177+0000","last_became_peered":"2024-04-27T07:04:15.629177+0000","last_unstale":"2024-04-27T07:04:17.495828+0000","last_undegraded":"2024-04-27T07:04:17.495828+0000","last_fullsized":"2024-04-27T07:04:17.495828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493333+0000","last_change":"2024-04-27T07:04:15.619427+0000","last_active":"2024-04-27T07:04:17.493333+0000","last_peered":"2024-04-27T07:04:17.493333+0000","last_clean":"2024-04-27T07:04:17.493333+0000","last_became_active":"2024-04-27T07:04:15.617987+0000","last_became_peered":"2024-04-27T07:04:15.617987+0000","last_unstale":"2024-04-27T07:04:17.493333+0000","last_undegraded":"2024-04-27T07:04:17.493333+0000","last_fullsized":"2024-04-27T07:04:17.493333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:39:04.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496704+0000","last_change":"2024-04-27T07:04:15.614547+0000","last_active":"2024-04-27T07:04:17.496704+0000","last_peered":"2024-04-27T07:04:17.496704+0000","last_clean":"2024-04-27T07:04:17.496704+0000","last_became_active":"2024-04-27T07:04:15.614210+0000","last_became_peered":"2024-04-27T07:04:15.614210+0000","last_unstale":"2024-04-27T07:04:17.496704+0000","last_undegraded":"2024-04-27T07:04:17.496704+0000","last_fullsized":"2024-04-27T07:04:17.496704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:40:20.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614461+0000","last_change":"2024-04-27T07:04:15.650072+0000","last_active":"2024-04-27T07:04:16.614461+0000","last_peered":"2024-04-27T07:04:16.614461+0000","last_clean":"2024-04-27T07:04:16.614461+0000","last_became_active":"2024-04-27T07:04:15.649461+0000","last_became_peered":"2024-04-27T07:04:15.649461+0000","last_unstale":"2024-04-27T07:04:16.614461+0000","last_undegraded":"2024-04-27T07:04:16.614461+0000","last_fullsized":"2024-04-27T07:04:16.614461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610542+0000","last_change":"2024-04-27T07:04:15.625742+0000","last_active":"2024-04-27T07:04:16.610542+0000","last_peered":"2024-04-27T07:04:16.610542+0000","last_clean":"2024-04-27T07:04:16.610542+0000","last_became_active":"2024-04-27T07:04:15.625049+0000","last_became_peered":"2024-04-27T07:04:15.625049+0000","last_unstale":"2024-04-27T07:04:16.610542+0000","last_undegraded":"2024-04-27T07:04:16.610542+0000","last_fullsized":"2024-04-27T07:04:16.610542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496813+0000","last_change":"2024-04-27T07:04:15.629376+0000","last_active":"2024-04-27T07:04:17.496813+0000","last_peered":"2024-04-27T07:04:17.496813+0000","last_clean":"2024-04-27T07:04:17.496813+0000","last_became_active":"2024-04-27T07:04:15.629058+0000","last_became_peered":"2024-04-27T07:04:15.629058+0000","last_unstale":"2024-04-27T07:04:17.496813+0000","last_undegraded":"2024-04-27T07:04:17.496813+0000","last_fullsized":"2024-04-27T07:04:17.496813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494539+0000","last_change":"2024-04-27T07:04:15.620883+0000","last_active":"2024-04-27T07:04:17.494539+0000","last_peered":"2024-04-27T07:04:17.494539+0000","last_clean":"2024-04-27T07:04:17.494539+0000","last_became_active":"2024-04-27T07:04:15.620549+0000","last_became_peered":"2024-04-27T07:04:15.620549+0000","last_unstale":"2024-04-27T07:04:17.494539+0000","last_undegraded":"2024-04-27T07:04:17.494539+0000","last_fullsized":"2024-04-27T07:04:17.494539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493778+0000","last_change":"2024-04-27T07:04:15.631001+0000","last_active":"2024-04-27T07:04:17.493778+0000","last_peered":"2024-04-27T07:04:17.493778+0000","last_clean":"2024-04-27T07:04:17.493778+0000","last_became_active":"2024-04-27T07:04:15.628448+0000","last_became_peered":"2024-04-27T07:04:15.628448+0000","last_unstale":"2024-04-27T07:04:17.493778+0000","last_undegraded":"2024-04-27T07:04:17.493778+0000","last_fullsized":"2024-04-27T07:04:17.493778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493518+0000","last_change":"2024-04-27T07:04:15.623687+0000","last_active":"2024-04-27T07:04:17.493518+0000","last_peered":"2024-04-27T07:04:17.493518+0000","last_clean":"2024-04-27T07:04:17.493518+0000","last_became_active":"2024-04-27T07:04:15.619092+0000","last_became_peered":"2024-04-27T07:04:15.619092+0000","last_unstale":"2024-04-27T07:04:17.493518+0000","last_undegraded":"2024-04-27T07:04:17.493518+0000","last_fullsized":"2024-04-27T07:04:17.493518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:50.960522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612331+0000","last_change":"2024-04-27T07:04:15.601950+0000","last_active":"2024-04-27T07:04:16.612331+0000","last_peered":"2024-04-27T07:04:16.612331+0000","last_clean":"2024-04-27T07:04:16.612331+0000","last_became_active":"2024-04-27T07:04:15.601144+0000","last_became_peered":"2024-04-27T07:04:15.601144+0000","last_unstale":"2024-04-27T07:04:16.612331+0000","last_undegraded":"2024-04-27T07:04:16.612331+0000","last_fullsized":"2024-04-27T07:04:16.612331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613553+0000","last_change":"2024-04-27T07:04:15.631943+0000","last_active":"2024-04-27T07:04:16.613553+0000","last_peered":"2024-04-27T07:04:16.613553+0000","last_clean":"2024-04-27T07:04:16.613553+0000","last_became_active":"2024-04-27T07:04:15.619207+0000","last_became_peered":"2024-04-27T07:04:15.619207+0000","last_unstale":"2024-04-27T07:04:16.613553+0000","last_undegraded":"2024-04-27T07:04:16.613553+0000","last_fullsized":"2024-04-27T07:04:16.613553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:58.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496502+0000","last_change":"2024-04-27T07:04:14.615664+0000","last_active":"2024-04-27T07:04:17.496502+0000","last_peered":"2024-04-27T07:04:17.496502+0000","last_clean":"2024-04-27T07:04:17.496502+0000","last_became_active":"2024-04-27T07:04:14.615220+0000","last_became_peered":"2024-04-27T07:04:14.615220+0000","last_unstale":"2024-04-27T07:04:17.496502+0000","last_undegraded":"2024-04-27T07:04:17.496502+0000","last_fullsized":"2024-04-27T07:04:17.496502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494747+0000","last_change":"2024-04-27T07:04:15.619876+0000","last_active":"2024-04-27T07:04:17.494747+0000","last_peered":"2024-04-27T07:04:17.494747+0000","last_clean":"2024-04-27T07:04:17.494747+0000","last_became_active":"2024-04-27T07:04:15.619131+0000","last_became_peered":"2024-04-27T07:04:15.619131+0000","last_unstale":"2024-04-27T07:04:17.494747+0000","last_undegraded":"2024-04-27T07:04:17.494747+0000","last_fullsized":"2024-04-27T07:04:17.494747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614112+0000","last_change":"2024-04-27T07:04:14.611792+0000","last_active":"2024-04-27T07:04:16.614112+0000","last_peered":"2024-04-27T07:04:16.614112+0000","last_clean":"2024-04-27T07:04:16.614112+0000","last_became_active":"2024-04-27T07:04:14.609221+0000","last_became_peered":"2024-04-27T07:04:14.609221+0000","last_unstale":"2024-04-27T07:04:16.614112+0000","last_undegraded":"2024-04-27T07:04:16.614112+0000","last_fullsized":"2024-04-27T07:04:16.614112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495969+0000","last_change":"2024-04-27T07:04:14.608168+0000","last_active":"2024-04-27T07:04:17.495969+0000","last_peered":"2024-04-27T07:04:17.495969+0000","last_clean":"2024-04-27T07:04:17.495969+0000","last_became_active":"2024-04-27T07:04:14.607735+0000","last_became_peered":"2024-04-27T07:04:14.607735+0000","last_unstale":"2024-04-27T07:04:17.495969+0000","last_undegraded":"2024-04-27T07:04:17.495969+0000","last_fullsized":"2024-04-27T07:04:17.495969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:06.395968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495767+0000","last_change":"2024-04-27T07:04:15.612016+0000","last_active":"2024-04-27T07:04:17.495767+0000","last_peered":"2024-04-27T07:04:17.495767+0000","last_clean":"2024-04-27T07:04:17.495767+0000","last_became_active":"2024-04-27T07:04:15.611658+0000","last_became_peered":"2024-04-27T07:04:15.611658+0000","last_unstale":"2024-04-27T07:04:17.495767+0000","last_undegraded":"2024-04-27T07:04:17.495767+0000","last_fullsized":"2024-04-27T07:04:17.495767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492906+0000","last_change":"2024-04-27T07:04:14.620476+0000","last_active":"2024-04-27T07:04:17.492906+0000","last_peered":"2024-04-27T07:04:17.492906+0000","last_clean":"2024-04-27T07:04:17.492906+0000","last_became_active":"2024-04-27T07:04:14.617893+0000","last_became_peered":"2024-04-27T07:04:14.617893+0000","last_unstale":"2024-04-27T07:04:17.492906+0000","last_undegraded":"2024-04-27T07:04:17.492906+0000","last_fullsized":"2024-04-27T07:04:17.492906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610222+0000","last_change":"2024-04-27T07:04:15.624591+0000","last_active":"2024-04-27T07:04:16.610222+0000","last_peered":"2024-04-27T07:04:16.610222+0000","last_clean":"2024-04-27T07:04:16.610222+0000","last_became_active":"2024-04-27T07:04:15.624299+0000","last_became_peered":"2024-04-27T07:04:15.624299+0000","last_unstale":"2024-04-27T07:04:16.610222+0000","last_undegraded":"2024-04-27T07:04:16.610222+0000","last_fullsized":"2024-04-27T07:04:16.610222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614771+0000","last_change":"2024-04-27T07:04:14.621378+0000","last_active":"2024-04-27T07:04:16.614771+0000","last_peered":"2024-04-27T07:04:16.614771+0000","last_clean":"2024-04-27T07:04:16.614771+0000","last_became_active":"2024-04-27T07:04:14.620678+0000","last_became_peered":"2024-04-27T07:04:14.620678+0000","last_unstale":"2024-04-27T07:04:16.614771+0000","last_undegraded":"2024-04-27T07:04:16.614771+0000","last_fullsized":"2024-04-27T07:04:16.614771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493724+0000","last_change":"2024-04-27T07:04:15.647396+0000","last_active":"2024-04-27T07:04:17.493724+0000","last_peered":"2024-04-27T07:04:17.493724+0000","last_clean":"2024-04-27T07:04:17.493724+0000","last_became_active":"2024-04-27T07:04:15.646774+0000","last_became_peered":"2024-04-27T07:04:15.646774+0000","last_unstale":"2024-04-27T07:04:17.493724+0000","last_undegraded":"2024-04-27T07:04:17.493724+0000","last_fullsized":"2024-04-27T07:04:17.493724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493229+0000","last_change":"2024-04-27T07:04:14.621935+0000","last_active":"2024-04-27T07:04:17.493229+0000","last_peered":"2024-04-27T07:04:17.493229+0000","last_clean":"2024-04-27T07:04:17.493229+0000","last_became_active":"2024-04-27T07:04:14.620120+0000","last_became_peered":"2024-04-27T07:04:14.620120+0000","last_unstale":"2024-04-27T07:04:17.493229+0000","last_undegraded":"2024-04-27T07:04:17.493229+0000","last_fullsized":"2024-04-27T07:04:17.493229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494443+0000","last_change":"2024-04-27T07:04:15.626451+0000","last_active":"2024-04-27T07:04:17.494443+0000","last_peered":"2024-04-27T07:04:17.494443+0000","last_clean":"2024-04-27T07:04:17.494443+0000","last_became_active":"2024-04-27T07:04:15.626147+0000","last_became_peered":"2024-04-27T07:04:15.626147+0000","last_unstale":"2024-04-27T07:04:17.494443+0000","last_undegraded":"2024-04-27T07:04:17.494443+0000","last_fullsized":"2024-04-27T07:04:17.494443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610433+0000","last_change":"2024-04-27T07:04:14.619618+0000","last_active":"2024-04-27T07:04:16.610433+0000","last_peered":"2024-04-27T07:04:16.610433+0000","last_clean":"2024-04-27T07:04:16.610433+0000","last_became_active":"2024-04-27T07:04:14.619155+0000","last_became_peered":"2024-04-27T07:04:14.619155+0000","last_unstale":"2024-04-27T07:04:16.610433+0000","last_undegraded":"2024-04-27T07:04:16.610433+0000","last_fullsized":"2024-04-27T07:04:16.610433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614234+0000","last_change":"2024-04-27T07:04:15.628583+0000","last_active":"2024-04-27T07:04:16.614234+0000","last_peered":"2024-04-27T07:04:16.614234+0000","last_clean":"2024-04-27T07:04:16.614234+0000","last_became_active":"2024-04-27T07:04:15.621801+0000","last_became_peered":"2024-04-27T07:04:15.621801+0000","last_unstale":"2024-04-27T07:04:16.614234+0000","last_undegraded":"2024-04-27T07:04:16.614234+0000","last_fullsized":"2024-04-27T07:04:16.614234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:37.376293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615199+0000","last_change":"2024-04-27T07:04:14.609138+0000","last_active":"2024-04-27T07:04:16.615199+0000","last_peered":"2024-04-27T07:04:16.615199+0000","last_clean":"2024-04-27T07:04:16.615199+0000","last_became_active":"2024-04-27T07:04:14.608374+0000","last_became_peered":"2024-04-27T07:04:14.608374+0000","last_unstale":"2024-04-27T07:04:16.615199+0000","last_undegraded":"2024-04-27T07:04:16.615199+0000","last_fullsized":"2024-04-27T07:04:16.615199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495184+0000","last_change":"2024-04-27T07:04:15.651834+0000","last_active":"2024-04-27T07:04:17.495184+0000","last_peered":"2024-04-27T07:04:17.495184+0000","last_clean":"2024-04-27T07:04:17.495184+0000","last_became_active":"2024-04-27T07:04:15.651286+0000","last_became_peered":"2024-04-27T07:04:15.651286+0000","last_unstale":"2024-04-27T07:04:17.495184+0000","last_undegraded":"2024-04-27T07:04:17.495184+0000","last_fullsized":"2024-04-27T07:04:17.495184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:00:38.115028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494797+0000","last_change":"2024-04-27T07:04:14.616280+0000","last_active":"2024-04-27T07:04:17.494797+0000","last_peered":"2024-04-27T07:04:17.494797+0000","last_clean":"2024-04-27T07:04:17.494797+0000","last_became_active":"2024-04-27T07:04:14.615920+0000","last_became_peered":"2024-04-27T07:04:14.615920+0000","last_unstale":"2024-04-27T07:04:17.494797+0000","last_undegraded":"2024-04-27T07:04:17.494797+0000","last_fullsized":"2024-04-27T07:04:17.494797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:06.395968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493937+0000","last_change":"2024-04-27T07:04:15.647042+0000","last_active":"2024-04-27T07:04:17.493937+0000","last_peered":"2024-04-27T07:04:17.493937+0000","last_clean":"2024-04-27T07:04:17.493937+0000","last_became_active":"2024-04-27T07:04:15.646683+0000","last_became_peered":"2024-04-27T07:04:15.646683+0000","last_unstale":"2024-04-27T07:04:17.493937+0000","last_undegraded":"2024-04-27T07:04:17.493937+0000","last_fullsized":"2024-04-27T07:04:17.493937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:05:55.398463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493914+0000","last_change":"2024-04-27T07:04:14.620284+0000","last_active":"2024-04-27T07:04:17.493914+0000","last_peered":"2024-04-27T07:04:17.493914+0000","last_clean":"2024-04-27T07:04:17.493914+0000","last_became_active":"2024-04-27T07:04:14.619379+0000","last_became_peered":"2024-04-27T07:04:14.619379+0000","last_unstale":"2024-04-27T07:04:17.493914+0000","last_undegraded":"2024-04-27T07:04:17.493914+0000","last_fullsized":"2024-04-27T07:04:17.493914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495014+0000","last_change":"2024-04-27T07:04:15.635161+0000","last_active":"2024-04-27T07:04:17.495014+0000","last_peered":"2024-04-27T07:04:17.495014+0000","last_clean":"2024-04-27T07:04:17.495014+0000","last_became_active":"2024-04-27T07:04:15.634423+0000","last_became_peered":"2024-04-27T07:04:15.634423+0000","last_unstale":"2024-04-27T07:04:17.495014+0000","last_undegraded":"2024-04-27T07:04:17.495014+0000","last_fullsized":"2024-04-27T07:04:17.495014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:17.226045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612693+0000","last_change":"2024-04-27T07:04:14.612975+0000","last_active":"2024-04-27T07:04:16.612693+0000","last_peered":"2024-04-27T07:04:16.612693+0000","last_clean":"2024-04-27T07:04:16.612693+0000","last_became_active":"2024-04-27T07:04:14.608937+0000","last_became_peered":"2024-04-27T07:04:14.608937+0000","last_unstale":"2024-04-27T07:04:16.612693+0000","last_undegraded":"2024-04-27T07:04:16.612693+0000","last_fullsized":"2024-04-27T07:04:16.612693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612667+0000","last_change":"2024-04-27T07:04:15.601024+0000","last_active":"2024-04-27T07:04:16.612667+0000","last_peered":"2024-04-27T07:04:16.612667+0000","last_clean":"2024-04-27T07:04:16.612667+0000","last_became_active":"2024-04-27T07:04:15.600245+0000","last_became_peered":"2024-04-27T07:04:15.600245+0000","last_unstale":"2024-04-27T07:04:16.612667+0000","last_undegraded":"2024-04-27T07:04:16.612667+0000","last_fullsized":"2024-04-27T07:04:16.612667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492667+0000","last_change":"2024-04-27T07:04:14.594793+0000","last_active":"2024-04-27T07:04:17.492667+0000","last_peered":"2024-04-27T07:04:17.492667+0000","last_clean":"2024-04-27T07:04:17.492667+0000","last_became_active":"2024-04-27T07:04:14.594022+0000","last_became_peered":"2024-04-27T07:04:14.594022+0000","last_unstale":"2024-04-27T07:04:17.492667+0000","last_undegraded":"2024-04-27T07:04:17.492667+0000","last_fullsized":"2024-04-27T07:04:17.492667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:01.729925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493727+0000","last_change":"2024-04-27T07:04:15.639932+0000","last_active":"2024-04-27T07:04:17.493727+0000","last_peered":"2024-04-27T07:04:17.493727+0000","last_clean":"2024-04-27T07:04:17.493727+0000","last_became_active":"2024-04-27T07:04:15.639192+0000","last_became_peered":"2024-04-27T07:04:15.639192+0000","last_unstale":"2024-04-27T07:04:17.493727+0000","last_undegraded":"2024-04-27T07:04:17.493727+0000","last_fullsized":"2024-04-27T07:04:17.493727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614105+0000","last_change":"2024-04-27T07:04:14.605930+0000","last_active":"2024-04-27T07:04:16.614105+0000","last_peered":"2024-04-27T07:04:16.614105+0000","last_clean":"2024-04-27T07:04:16.614105+0000","last_became_active":"2024-04-27T07:04:14.604957+0000","last_became_peered":"2024-04-27T07:04:14.604957+0000","last_unstale":"2024-04-27T07:04:16.614105+0000","last_undegraded":"2024-04-27T07:04:16.614105+0000","last_fullsized":"2024-04-27T07:04:16.614105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:15.465739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495558+0000","last_change":"2024-04-27T07:04:15.635261+0000","last_active":"2024-04-27T07:04:17.495558+0000","last_peered":"2024-04-27T07:04:17.495558+0000","last_clean":"2024-04-27T07:04:17.495558+0000","last_became_active":"2024-04-27T07:04:15.634020+0000","last_became_peered":"2024-04-27T07:04:15.634020+0000","last_unstale":"2024-04-27T07:04:17.495558+0000","last_undegraded":"2024-04-27T07:04:17.495558+0000","last_fullsized":"2024-04-27T07:04:17.495558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:08.890952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493458+0000","last_change":"2024-04-27T07:04:14.600680+0000","last_active":"2024-04-27T07:04:17.493458+0000","last_peered":"2024-04-27T07:04:17.493458+0000","last_clean":"2024-04-27T07:04:17.493458+0000","last_became_active":"2024-04-27T07:04:14.600233+0000","last_became_peered":"2024-04-27T07:04:14.600233+0000","last_unstale":"2024-04-27T07:04:17.493458+0000","last_undegraded":"2024-04-27T07:04:17.493458+0000","last_fullsized":"2024-04-27T07:04:17.493458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495918+0000","last_change":"2024-04-27T07:04:15.643279+0000","last_active":"2024-04-27T07:04:17.495918+0000","last_peered":"2024-04-27T07:04:17.495918+0000","last_clean":"2024-04-27T07:04:17.495918+0000","last_became_active":"2024-04-27T07:04:15.641303+0000","last_became_peered":"2024-04-27T07:04:15.641303+0000","last_unstale":"2024-04-27T07:04:17.495918+0000","last_undegraded":"2024-04-27T07:04:17.495918+0000","last_fullsized":"2024-04-27T07:04:17.495918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:02:46.002992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610599+0000","last_change":"2024-04-27T07:04:14.617808+0000","last_active":"2024-04-27T07:04:16.610599+0000","last_peered":"2024-04-27T07:04:16.610599+0000","last_clean":"2024-04-27T07:04:16.610599+0000","last_became_active":"2024-04-27T07:04:14.617505+0000","last_became_peered":"2024-04-27T07:04:14.617505+0000","last_unstale":"2024-04-27T07:04:16.610599+0000","last_undegraded":"2024-04-27T07:04:16.610599+0000","last_fullsized":"2024-04-27T07:04:16.610599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494318+0000","last_change":"2024-04-27T07:04:15.619586+0000","last_active":"2024-04-27T07:04:17.494318+0000","last_peered":"2024-04-27T07:04:17.494318+0000","last_clean":"2024-04-27T07:04:17.494318+0000","last_became_active":"2024-04-27T07:04:15.617842+0000","last_became_peered":"2024-04-27T07:04:15.617842+0000","last_unstale":"2024-04-27T07:04:17.494318+0000","last_undegraded":"2024-04-27T07:04:17.494318+0000","last_fullsized":"2024-04-27T07:04:17.494318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:46.491888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493601+0000","last_change":"2024-04-27T07:04:14.619149+0000","last_active":"2024-04-27T07:04:17.493601+0000","last_peered":"2024-04-27T07:04:17.493601+0000","last_clean":"2024-04-27T07:04:17.493601+0000","last_became_active":"2024-04-27T07:04:14.618660+0000","last_became_peered":"2024-04-27T07:04:14.618660+0000","last_unstale":"2024-04-27T07:04:17.493601+0000","last_undegraded":"2024-04-27T07:04:17.493601+0000","last_fullsized":"2024-04-27T07:04:17.493601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495600+0000","last_change":"2024-04-27T07:04:15.640980+0000","last_active":"2024-04-27T07:04:17.495600+0000","last_peered":"2024-04-27T07:04:17.495600+0000","last_clean":"2024-04-27T07:04:17.495600+0000","last_became_active":"2024-04-27T07:04:15.640289+0000","last_became_peered":"2024-04-27T07:04:15.640289+0000","last_unstale":"2024-04-27T07:04:17.495600+0000","last_undegraded":"2024-04-27T07:04:17.495600+0000","last_fullsized":"2024-04-27T07:04:17.495600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:19.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492606+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:04:17.492606+0000","last_peered":"2024-04-27T07:04:17.492606+0000","last_clean":"2024-04-27T07:04:17.492606+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:04:17.492606+0000","last_undegraded":"2024-04-27T07:04:17.492606+0000","last_fullsized":"2024-04-27T07:04:17.492606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493168+0000","last_change":"2024-04-27T07:04:14.621914+0000","last_active":"2024-04-27T07:04:17.493168+0000","last_peered":"2024-04-27T07:04:17.493168+0000","last_clean":"2024-04-27T07:04:17.493168+0000","last_became_active":"2024-04-27T07:04:14.620901+0000","last_became_peered":"2024-04-27T07:04:14.620901+0000","last_unstale":"2024-04-27T07:04:17.493168+0000","last_undegraded":"2024-04-27T07:04:17.493168+0000","last_fullsized":"2024-04-27T07:04:17.493168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496303+0000","last_change":"2024-04-27T07:04:15.635276+0000","last_active":"2024-04-27T07:04:17.496303+0000","last_peered":"2024-04-27T07:04:17.496303+0000","last_clean":"2024-04-27T07:04:17.496303+0000","last_became_active":"2024-04-27T07:04:15.634946+0000","last_became_peered":"2024-04-27T07:04:15.634946+0000","last_unstale":"2024-04-27T07:04:17.496303+0000","last_undegraded":"2024-04-27T07:04:17.496303+0000","last_fullsized":"2024-04-27T07:04:17.496303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492322+0000","last_change":"2024-04-27T07:04:14.628608+0000","last_active":"2024-04-27T07:04:17.492322+0000","last_peered":"2024-04-27T07:04:17.492322+0000","last_clean":"2024-04-27T07:04:17.492322+0000","last_became_active":"2024-04-27T07:04:14.626795+0000","last_became_peered":"2024-04-27T07:04:14.626795+0000","last_unstale":"2024-04-27T07:04:17.492322+0000","last_undegraded":"2024-04-27T07:04:17.492322+0000","last_fullsized":"2024-04-27T07:04:17.492322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:32:26.049320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493125+0000","last_change":"2024-04-27T07:04:15.604705+0000","last_active":"2024-04-27T07:04:17.493125+0000","last_peered":"2024-04-27T07:04:17.493125+0000","last_clean":"2024-04-27T07:04:17.493125+0000","last_became_active":"2024-04-27T07:04:15.603226+0000","last_became_peered":"2024-04-27T07:04:15.603226+0000","last_unstale":"2024-04-27T07:04:17.493125+0000","last_undegraded":"2024-04-27T07:04:17.493125+0000","last_fullsized":"2024-04-27T07:04:17.493125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:24.246666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492975+0000","last_change":"2024-04-27T07:04:14.601261+0000","last_active":"2024-04-27T07:04:17.492975+0000","last_peered":"2024-04-27T07:04:17.492975+0000","last_clean":"2024-04-27T07:04:17.492975+0000","last_became_active":"2024-04-27T07:04:14.600555+0000","last_became_peered":"2024-04-27T07:04:14.600555+0000","last_unstale":"2024-04-27T07:04:17.492975+0000","last_undegraded":"2024-04-27T07:04:17.492975+0000","last_fullsized":"2024-04-27T07:04:17.492975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610538+0000","last_change":"2024-04-27T07:04:15.632176+0000","last_active":"2024-04-27T07:04:16.610538+0000","last_peered":"2024-04-27T07:04:16.610538+0000","last_clean":"2024-04-27T07:04:16.610538+0000","last_became_active":"2024-04-27T07:04:15.631861+0000","last_became_peered":"2024-04-27T07:04:15.631861+0000","last_unstale":"2024-04-27T07:04:16.610538+0000","last_undegraded":"2024-04-27T07:04:16.610538+0000","last_fullsized":"2024-04-27T07:04:16.610538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494374+0000","last_change":"2024-04-27T07:04:14.623584+0000","last_active":"2024-04-27T07:04:17.494374+0000","last_peered":"2024-04-27T07:04:17.494374+0000","last_clean":"2024-04-27T07:04:17.494374+0000","last_became_active":"2024-04-27T07:04:14.620709+0000","last_became_peered":"2024-04-27T07:04:14.620709+0000","last_unstale":"2024-04-27T07:04:17.494374+0000","last_undegraded":"2024-04-27T07:04:17.494374+0000","last_fullsized":"2024-04-27T07:04:17.494374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614757+0000","last_change":"2024-04-27T07:04:15.633827+0000","last_active":"2024-04-27T07:04:16.614757+0000","last_peered":"2024-04-27T07:04:16.614757+0000","last_clean":"2024-04-27T07:04:16.614757+0000","last_became_active":"2024-04-27T07:04:15.632912+0000","last_became_peered":"2024-04-27T07:04:15.632912+0000","last_unstale":"2024-04-27T07:04:16.614757+0000","last_undegraded":"2024-04-27T07:04:16.614757+0000","last_fullsized":"2024-04-27T07:04:16.614757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:50.960522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610382+0000","last_change":"2024-04-27T07:04:14.603584+0000","last_active":"2024-04-27T07:04:16.610382+0000","last_peered":"2024-04-27T07:04:16.610382+0000","last_clean":"2024-04-27T07:04:16.610382+0000","last_became_active":"2024-04-27T07:04:14.602186+0000","last_became_peered":"2024-04-27T07:04:14.602186+0000","last_unstale":"2024-04-27T07:04:16.610382+0000","last_undegraded":"2024-04-27T07:04:16.610382+0000","last_fullsized":"2024-04-27T07:04:16.610382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494048+0000","last_change":"2024-04-27T07:04:15.620627+0000","last_active":"2024-04-27T07:04:17.494048+0000","last_peered":"2024-04-27T07:04:17.494048+0000","last_clean":"2024-04-27T07:04:17.494048+0000","last_became_active":"2024-04-27T07:04:15.619725+0000","last_became_peered":"2024-04-27T07:04:15.619725+0000","last_unstale":"2024-04-27T07:04:17.494048+0000","last_undegraded":"2024-04-27T07:04:17.494048+0000","last_fullsized":"2024-04-27T07:04:17.494048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:17.226045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495737+0000","last_change":"2024-04-27T07:04:14.630298+0000","last_active":"2024-04-27T07:04:17.495737+0000","last_peered":"2024-04-27T07:04:17.495737+0000","last_clean":"2024-04-27T07:04:17.495737+0000","last_became_active":"2024-04-27T07:04:14.629539+0000","last_became_peered":"2024-04-27T07:04:14.629539+0000","last_unstale":"2024-04-27T07:04:17.495737+0000","last_undegraded":"2024-04-27T07:04:17.495737+0000","last_fullsized":"2024-04-27T07:04:17.495737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:44.902463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610300+0000","last_change":"2024-04-27T07:04:15.604054+0000","last_active":"2024-04-27T07:04:16.610300+0000","last_peered":"2024-04-27T07:04:16.610300+0000","last_clean":"2024-04-27T07:04:16.610300+0000","last_became_active":"2024-04-27T07:04:15.603243+0000","last_became_peered":"2024-04-27T07:04:15.603243+0000","last_unstale":"2024-04-27T07:04:16.610300+0000","last_undegraded":"2024-04-27T07:04:16.610300+0000","last_fullsized":"2024-04-27T07:04:16.610300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610793+0000","last_change":"2024-04-27T07:04:14.614799+0000","last_active":"2024-04-27T07:04:16.610793+0000","last_peered":"2024-04-27T07:04:16.610793+0000","last_clean":"2024-04-27T07:04:16.610793+0000","last_became_active":"2024-04-27T07:04:14.614348+0000","last_became_peered":"2024-04-27T07:04:14.614348+0000","last_unstale":"2024-04-27T07:04:16.610793+0000","last_undegraded":"2024-04-27T07:04:16.610793+0000","last_fullsized":"2024-04-27T07:04:16.610793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:49.948227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.497067+0000","last_change":"2024-04-27T07:04:15.649374+0000","last_active":"2024-04-27T07:04:17.497067+0000","last_peered":"2024-04-27T07:04:17.497067+0000","last_clean":"2024-04-27T07:04:17.497067+0000","last_became_active":"2024-04-27T07:04:15.649079+0000","last_became_peered":"2024-04-27T07:04:15.649079+0000","last_unstale":"2024-04-27T07:04:17.497067+0000","last_undegraded":"2024-04-27T07:04:17.497067+0000","last_fullsized":"2024-04-27T07:04:17.497067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615045+0000","last_change":"2024-04-27T07:04:14.630929+0000","last_active":"2024-04-27T07:04:16.615045+0000","last_peered":"2024-04-27T07:04:16.615045+0000","last_clean":"2024-04-27T07:04:16.615045+0000","last_became_active":"2024-04-27T07:04:14.630345+0000","last_became_peered":"2024-04-27T07:04:14.630345+0000","last_unstale":"2024-04-27T07:04:16.615045+0000","last_undegraded":"2024-04-27T07:04:16.615045+0000","last_fullsized":"2024-04-27T07:04:16.615045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494516+0000","last_change":"2024-04-27T07:04:15.605948+0000","last_active":"2024-04-27T07:04:17.494516+0000","last_peered":"2024-04-27T07:04:17.494516+0000","last_clean":"2024-04-27T07:04:17.494516+0000","last_became_active":"2024-04-27T07:04:15.603763+0000","last_became_peered":"2024-04-27T07:04:15.603763+0000","last_unstale":"2024-04-27T07:04:17.494516+0000","last_undegraded":"2024-04-27T07:04:17.494516+0000","last_fullsized":"2024-04-27T07:04:17.494516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492302+0000","last_change":"2024-04-27T07:04:14.628661+0000","last_active":"2024-04-27T07:04:17.492302+0000","last_peered":"2024-04-27T07:04:17.492302+0000","last_clean":"2024-04-27T07:04:17.492302+0000","last_became_active":"2024-04-27T07:04:14.626731+0000","last_became_peered":"2024-04-27T07:04:14.626731+0000","last_unstale":"2024-04-27T07:04:17.492302+0000","last_undegraded":"2024-04-27T07:04:17.492302+0000","last_fullsized":"2024-04-27T07:04:17.492302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613464+0000","last_change":"2024-04-27T07:04:15.608033+0000","last_active":"2024-04-27T07:04:16.613464+0000","last_peered":"2024-04-27T07:04:16.613464+0000","last_clean":"2024-04-27T07:04:16.613464+0000","last_became_active":"2024-04-27T07:04:15.607219+0000","last_became_peered":"2024-04-27T07:04:15.607219+0000","last_unstale":"2024-04-27T07:04:16.613464+0000","last_undegraded":"2024-04-27T07:04:16.613464+0000","last_fullsized":"2024-04-27T07:04:16.613464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:02.140237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610281+0000","last_change":"2024-04-27T07:04:14.613406+0000","last_active":"2024-04-27T07:04:16.610281+0000","last_peered":"2024-04-27T07:04:16.610281+0000","last_clean":"2024-04-27T07:04:16.610281+0000","last_became_active":"2024-04-27T07:04:14.613017+0000","last_became_peered":"2024-04-27T07:04:14.613017+0000","last_unstale":"2024-04-27T07:04:16.610281+0000","last_undegraded":"2024-04-27T07:04:16.610281+0000","last_fullsized":"2024-04-27T07:04:16.610281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496138+0000","last_change":"2024-04-27T07:04:15.617597+0000","last_active":"2024-04-27T07:04:17.496138+0000","last_peered":"2024-04-27T07:04:17.496138+0000","last_clean":"2024-04-27T07:04:17.496138+0000","last_became_active":"2024-04-27T07:04:15.616122+0000","last_became_peered":"2024-04-27T07:04:15.616122+0000","last_unstale":"2024-04-27T07:04:17.496138+0000","last_undegraded":"2024-04-27T07:04:17.496138+0000","last_fullsized":"2024-04-27T07:04:17.496138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610542+0000","last_change":"2024-04-27T07:04:14.604022+0000","last_active":"2024-04-27T07:04:16.610542+0000","last_peered":"2024-04-27T07:04:16.610542+0000","last_clean":"2024-04-27T07:04:16.610542+0000","last_became_active":"2024-04-27T07:04:14.603308+0000","last_became_peered":"2024-04-27T07:04:14.603308+0000","last_unstale":"2024-04-27T07:04:16.610542+0000","last_undegraded":"2024-04-27T07:04:16.610542+0000","last_fullsized":"2024-04-27T07:04:16.610542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496561+0000","last_change":"2024-04-27T07:04:15.620378+0000","last_active":"2024-04-27T07:04:17.496561+0000","last_peered":"2024-04-27T07:04:17.496561+0000","last_clean":"2024-04-27T07:04:17.496561+0000","last_became_active":"2024-04-27T07:04:15.620100+0000","last_became_peered":"2024-04-27T07:04:15.620100+0000","last_unstale":"2024-04-27T07:04:17.496561+0000","last_undegraded":"2024-04-27T07:04:17.496561+0000","last_fullsized":"2024-04-27T07:04:17.496561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494886+0000","last_change":"2024-04-27T07:04:14.629851+0000","last_active":"2024-04-27T07:04:17.494886+0000","last_peered":"2024-04-27T07:04:17.494886+0000","last_clean":"2024-04-27T07:04:17.494886+0000","last_became_active":"2024-04-27T07:04:14.629063+0000","last_became_peered":"2024-04-27T07:04:14.629063+0000","last_unstale":"2024-04-27T07:04:17.494886+0000","last_undegraded":"2024-04-27T07:04:17.494886+0000","last_fullsized":"2024-04-27T07:04:17.494886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:49.948227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613962+0000","last_change":"2024-04-27T07:04:15.616289+0000","last_active":"2024-04-27T07:04:16.613962+0000","last_peered":"2024-04-27T07:04:16.613962+0000","last_clean":"2024-04-27T07:04:16.613962+0000","last_became_active":"2024-04-27T07:04:15.615386+0000","last_became_peered":"2024-04-27T07:04:15.615386+0000","last_unstale":"2024-04-27T07:04:16.613962+0000","last_undegraded":"2024-04-27T07:04:16.613962+0000","last_fullsized":"2024-04-27T07:04:16.613962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493841+0000","last_change":"2024-04-27T07:04:15.646050+0000","last_active":"2024-04-27T07:04:17.493841+0000","last_peered":"2024-04-27T07:04:17.493841+0000","last_clean":"2024-04-27T07:04:17.493841+0000","last_became_active":"2024-04-27T07:04:15.645284+0000","last_became_peered":"2024-04-27T07:04:15.645284+0000","last_unstale":"2024-04-27T07:04:17.493841+0000","last_undegraded":"2024-04-27T07:04:17.493841+0000","last_fullsized":"2024-04-27T07:04:17.493841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:11.386203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494836+0000","last_change":"2024-04-27T07:04:14.629860+0000","last_active":"2024-04-27T07:04:17.494836+0000","last_peered":"2024-04-27T07:04:17.494836+0000","last_clean":"2024-04-27T07:04:17.494836+0000","last_became_active":"2024-04-27T07:04:14.629138+0000","last_became_peered":"2024-04-27T07:04:14.629138+0000","last_unstale":"2024-04-27T07:04:17.494836+0000","last_undegraded":"2024-04-27T07:04:17.494836+0000","last_fullsized":"2024-04-27T07:04:17.494836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494061+0000","last_change":"2024-04-27T07:04:15.638833+0000","last_active":"2024-04-27T07:04:17.494061+0000","last_peered":"2024-04-27T07:04:17.494061+0000","last_clean":"2024-04-27T07:04:17.494061+0000","last_became_active":"2024-04-27T07:04:15.638192+0000","last_became_peered":"2024-04-27T07:04:15.638192+0000","last_unstale":"2024-04-27T07:04:17.494061+0000","last_undegraded":"2024-04-27T07:04:17.494061+0000","last_fullsized":"2024-04-27T07:04:17.494061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:00:38.115028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495394+0000","last_change":"2024-04-27T07:04:14.632528+0000","last_active":"2024-04-27T07:04:17.495394+0000","last_peered":"2024-04-27T07:04:17.495394+0000","last_clean":"2024-04-27T07:04:17.495394+0000","last_became_active":"2024-04-27T07:04:14.631089+0000","last_became_peered":"2024-04-27T07:04:14.631089+0000","last_unstale":"2024-04-27T07:04:17.495394+0000","last_undegraded":"2024-04-27T07:04:17.495394+0000","last_fullsized":"2024-04-27T07:04:17.495394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:03:50.734904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494459+0000","last_change":"2024-04-27T07:04:15.626754+0000","last_active":"2024-04-27T07:04:17.494459+0000","last_peered":"2024-04-27T07:04:17.494459+0000","last_clean":"2024-04-27T07:04:17.494459+0000","last_became_active":"2024-04-27T07:04:15.626499+0000","last_became_peered":"2024-04-27T07:04:15.626499+0000","last_unstale":"2024-04-27T07:04:17.494459+0000","last_undegraded":"2024-04-27T07:04:17.494459+0000","last_fullsized":"2024-04-27T07:04:17.494459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496732+0000","last_change":"2024-04-27T07:04:14.598918+0000","last_active":"2024-04-27T07:04:17.496732+0000","last_peered":"2024-04-27T07:04:17.496732+0000","last_clean":"2024-04-27T07:04:17.496732+0000","last_became_active":"2024-04-27T07:04:14.598504+0000","last_became_peered":"2024-04-27T07:04:14.598504+0000","last_unstale":"2024-04-27T07:04:17.496732+0000","last_undegraded":"2024-04-27T07:04:17.496732+0000","last_fullsized":"2024-04-27T07:04:17.496732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:01.729925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496574+0000","last_change":"2024-04-27T07:04:15.625377+0000","last_active":"2024-04-27T07:04:17.496574+0000","last_peered":"2024-04-27T07:04:17.496574+0000","last_clean":"2024-04-27T07:04:17.496574+0000","last_became_active":"2024-04-27T07:04:15.625075+0000","last_became_peered":"2024-04-27T07:04:15.625075+0000","last_unstale":"2024-04-27T07:04:17.496574+0000","last_undegraded":"2024-04-27T07:04:17.496574+0000","last_fullsized":"2024-04-27T07:04:17.496574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:39:04.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494529+0000","last_change":"2024-04-27T07:04:14.626854+0000","last_active":"2024-04-27T07:04:17.494529+0000","last_peered":"2024-04-27T07:04:17.494529+0000","last_clean":"2024-04-27T07:04:17.494529+0000","last_became_active":"2024-04-27T07:04:14.626588+0000","last_became_peered":"2024-04-27T07:04:14.626588+0000","last_unstale":"2024-04-27T07:04:17.494529+0000","last_undegraded":"2024-04-27T07:04:17.494529+0000","last_fullsized":"2024-04-27T07:04:17.494529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:21.930932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:04:23.848 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:04:23.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:04:24.156 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:04:24.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.281+0000 7f7daaa9c640 1 Processor -- start 2024-04-27T07:04:24.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.281+0000 7f7daaa9c640 1 -- start start 2024-04-27T07:04:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7daaa9c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7da40b9a10 0x7f7da40b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7daaa9c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7da40b3680 0x7f7da4145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7daaa9c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7da406dac0 con 0x7f7da41460b0 2024-04-27T07:04:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7daaa9c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f7da406e890 con 0x7f7da40b9a10 2024-04-27T07:04:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7daaa9c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7da40b9df0 con 0x7f7da40b3680 2024-04-27T07:04:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da8838640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7da40b9a10 0x7f7da40b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da8838640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7da40b9a10 0x7f7da40b65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42000/0 (socket says 172.21.15.12:42000) 2024-04-27T07:04:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da8838640 1 -- 172.21.15.12:0/1486510167 learned_addr learned my addr 172.21.15.12:0/1486510167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da3fff640 1 --2- 172.21.15.12:0/1486510167 >> v2:172.21.15.53:3300/0 conn(0x7f7da40b3680 0x7f7da4145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da8838640 1 -- 172.21.15.12:0/1486510167 >> v2:172.21.15.53:3300/0 conn(0x7f7da40b3680 msgr2=0x7f7da4145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da8838640 1 --2- 172.21.15.12:0/1486510167 >> v2:172.21.15.53:3300/0 conn(0x7f7da40b3680 0x7f7da4145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da8838640 1 -- 172.21.15.12:0/1486510167 >> v1:172.21.15.53:6789/0 conn(0x7f7da41460b0 legacy=0x7f7da41490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da8838640 1 -- 172.21.15.12:0/1486510167 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da406e090 con 0x7f7da40b9a10 2024-04-27T07:04:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da8838640 1 --2- 172.21.15.12:0/1486510167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b9a10 0x7f7da40b65c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7da406f030 tx=0x7f7d9c01c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc495032917b2ed server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da37fe640 1 -- 172.21.15.12:0/1486510167 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0a6040 con 0x7f7da40b9a10 2024-04-27T07:04:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da37fe640 1 -- 172.21.15.12:0/1486510167 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c0af070 con 0x7f7da40b9a10 2024-04-27T07:04:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.285+0000 7f7da37fe640 1 -- 172.21.15.12:0/1486510167 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0aa660 con 0x7f7da40b9a10 2024-04-27T07:04:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- 172.21.15.12:0/1486510167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b9a10 msgr2=0x7f7da40b65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 --2- 172.21.15.12:0/1486510167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b9a10 0x7f7da40b65c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7da406f030 tx=0x7f7d9c01c6b0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- 172.21.15.12:0/1486510167 shutdown_connections 2024-04-27T07:04:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 --2- 172.21.15.12:0/1486510167 >> v2:172.21.15.53:3300/0 conn(0x7f7da40b3680 0x7f7da4145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 --2- 172.21.15.12:0/1486510167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b9a10 0x7f7da40b65c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- 172.21.15.12:0/1486510167 >> 172.21.15.12:0/1486510167 conn(0x7f7da4067d10 msgr2=0x7f7da4068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- 172.21.15.12:0/1486510167 shutdown_connections 2024-04-27T07:04:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- 172.21.15.12:0/1486510167 wait complete. 2024-04-27T07:04:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 Processor -- start 2024-04-27T07:04:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- start start 2024-04-27T07:04:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b3680 0x7f7da414ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7da40b9a10 0x7f7da41e0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7da41460b0 0x7f7da41e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7da40b6b00 con 0x7f7da40b3680 2024-04-27T07:04:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7da40740b0 con 0x7f7da41460b0 2024-04-27T07:04:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7da40b6c80 con 0x7f7da40b9a10 2024-04-27T07:04:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da8838640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b3680 0x7f7da414ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da8838640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b3680 0x7f7da414ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42002/0 (socket says 172.21.15.12:42002) 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da8838640 1 -- 172.21.15.12:0/2466590520 learned_addr learned my addr 172.21.15.12:0/2466590520 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da3fff640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7da40b9a10 0x7f7da41e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da9039640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7da41460b0 0x7f7da41e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da8838640 1 -- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7da40b9a10 msgr2=0x7f7da41e0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da8838640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7da40b9a10 0x7f7da41e0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da8838640 1 -- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7da41460b0 msgr2=0x7f7da41e0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da8838640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7da41460b0 0x7f7da41e0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da8838640 1 -- 172.21.15.12:0/2466590520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da414ff50 con 0x7f7da40b3680 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da8838640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b3680 0x7f7da414ed30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c01ad60 tx=0x7f7d9c019450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da17fa640 1 -- 172.21.15.12:0/2466590520 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0b9070 con 0x7f7da40b3680 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da17fa640 1 -- 172.21.15.12:0/2466590520 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c0aa700 con 0x7f7da40b3680 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- 172.21.15.12:0/2466590520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da41437d0 con 0x7f7da40b3680 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7da17fa640 1 -- 172.21.15.12:0/2466590520 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0aaad0 con 0x7f7da40b3680 2024-04-27T07:04:24.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.289+0000 7f7daaa9c640 1 -- 172.21.15.12:0/2466590520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da4140950 con 0x7f7da40b3680 2024-04-27T07:04:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.293+0000 7f7daaa9c640 1 -- 172.21.15.12:0/2466590520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da4150120 con 0x7f7da40b3680 2024-04-27T07:04:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.293+0000 7f7da17fa640 1 -- 172.21.15.12:0/2466590520 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7d9c0aac70 con 0x7f7da40b3680 2024-04-27T07:04:24.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.293+0000 7f7da17fa640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7d84068380 0x7f7d8406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.293+0000 7f7da17fa640 1 -- 172.21.15.12:0/2466590520 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7d9c120be0 con 0x7f7da40b3680 2024-04-27T07:04:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.293+0000 7f7da3fff640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7d84068380 0x7f7d8406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.293+0000 7f7da3fff640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7d84068380 0x7f7d8406a840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7da40b2a80 tx=0x7f7d8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:24.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.297+0000 7f7da17fa640 1 -- 172.21.15.12:0/2466590520 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7da406dac0 con 0x7f7da40b3680 2024-04-27T07:04:24.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.457+0000 7f7daaa9c640 1 -- 172.21.15.12:0/2466590520 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7da4068c20 con 0x7f7d84068380 2024-04-27T07:04:24.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.473+0000 7f7da17fa640 1 -- 172.21.15.12:0/2466590520 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f7da4068c20 con 0x7f7d84068380 2024-04-27T07:04:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:24.484 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.477+0000 7f7daaa9c640 1 -- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7d84068380 msgr2=0x7f7d8406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.477+0000 7f7daaa9c640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7d84068380 0x7f7d8406a840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7da40b2a80 tx=0x7f7d8c008040 comp rx=0 tx=0).stop 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.477+0000 7f7daaa9c640 1 -- 172.21.15.12:0/2466590520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b3680 msgr2=0x7f7da414ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.477+0000 7f7daaa9c640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b3680 0x7f7da414ed30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c01ad60 tx=0x7f7d9c019450 comp rx=0 tx=0).stop 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.477+0000 7f7daaa9c640 1 -- 172.21.15.12:0/2466590520 shutdown_connections 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.477+0000 7f7daaa9c640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7d84068380 0x7f7d8406a840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.477+0000 7f7daaa9c640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7da41460b0 0x7f7da41e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.477+0000 7f7daaa9c640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7da40b9a10 0x7f7da41e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.477+0000 7f7daaa9c640 1 --2- 172.21.15.12:0/2466590520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da40b3680 0x7f7da414ed30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.477+0000 7f7daaa9c640 1 -- 172.21.15.12:0/2466590520 >> 172.21.15.12:0/2466590520 conn(0x7f7da4067d10 msgr2=0x7f7da40b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.481+0000 7f7daaa9c640 1 -- 172.21.15.12:0/2466590520 shutdown_connections 2024-04-27T07:04:24.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.481+0000 7f7daaa9c640 1 -- 172.21.15.12:0/2466590520 wait complete. 2024-04-27T07:04:24.502 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-27T07:04:22.610469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235636,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739084,"statfs":{"total":773094113280,"available":772852822016,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003169"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495786+0000","last_change":"2024-04-27T07:04:15.652893+0000","last_active":"2024-04-27T07:04:17.495786+0000","last_peered":"2024-04-27T07:04:17.495786+0000","last_clean":"2024-04-27T07:04:17.495786+0000","last_became_active":"2024-04-27T07:04:15.652155+0000","last_became_peered":"2024-04-27T07:04:15.652155+0000","last_unstale":"2024-04-27T07:04:17.495786+0000","last_undegraded":"2024-04-27T07:04:17.495786+0000","last_fullsized":"2024-04-27T07:04:17.495786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:07:07.313585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610741+0000","last_change":"2024-04-27T07:04:15.622035+0000","last_active":"2024-04-27T07:04:16.610741+0000","last_peered":"2024-04-27T07:04:16.610741+0000","last_clean":"2024-04-27T07:04:16.610741+0000","last_became_active":"2024-04-27T07:04:15.621365+0000","last_became_peered":"2024-04-27T07:04:15.621365+0000","last_unstale":"2024-04-27T07:04:16.610741+0000","last_undegraded":"2024-04-27T07:04:16.610741+0000","last_fullsized":"2024-04-27T07:04:16.610741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494175+0000","last_change":"2024-04-27T07:04:15.603108+0000","last_active":"2024-04-27T07:04:17.494175+0000","last_peered":"2024-04-27T07:04:17.494175+0000","last_clean":"2024-04-27T07:04:17.494175+0000","last_became_active":"2024-04-27T07:04:15.602672+0000","last_became_peered":"2024-04-27T07:04:15.602672+0000","last_unstale":"2024-04-27T07:04:17.494175+0000","last_undegraded":"2024-04-27T07:04:17.494175+0000","last_fullsized":"2024-04-27T07:04:17.494175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494169+0000","last_change":"2024-04-27T07:04:15.621096+0000","last_active":"2024-04-27T07:04:17.494169+0000","last_peered":"2024-04-27T07:04:17.494169+0000","last_clean":"2024-04-27T07:04:17.494169+0000","last_became_active":"2024-04-27T07:04:15.618832+0000","last_became_peered":"2024-04-27T07:04:15.618832+0000","last_unstale":"2024-04-27T07:04:17.494169+0000","last_undegraded":"2024-04-27T07:04:17.494169+0000","last_fullsized":"2024-04-27T07:04:17.494169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:08.890952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494305+0000","last_change":"2024-04-27T07:04:15.627875+0000","last_active":"2024-04-27T07:04:17.494305+0000","last_peered":"2024-04-27T07:04:17.494305+0000","last_clean":"2024-04-27T07:04:17.494305+0000","last_became_active":"2024-04-27T07:04:15.626904+0000","last_became_peered":"2024-04-27T07:04:15.626904+0000","last_unstale":"2024-04-27T07:04:17.494305+0000","last_undegraded":"2024-04-27T07:04:17.494305+0000","last_fullsized":"2024-04-27T07:04:17.494305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:11.386203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494240+0000","last_change":"2024-04-27T07:04:15.632294+0000","last_active":"2024-04-27T07:04:17.494240+0000","last_peered":"2024-04-27T07:04:17.494240+0000","last_clean":"2024-04-27T07:04:17.494240+0000","last_became_active":"2024-04-27T07:04:15.630191+0000","last_became_peered":"2024-04-27T07:04:15.630191+0000","last_unstale":"2024-04-27T07:04:17.494240+0000","last_undegraded":"2024-04-27T07:04:17.494240+0000","last_fullsized":"2024-04-27T07:04:17.494240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:19.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493321+0000","last_change":"2024-04-27T07:04:15.652899+0000","last_active":"2024-04-27T07:04:17.493321+0000","last_peered":"2024-04-27T07:04:17.493321+0000","last_clean":"2024-04-27T07:04:17.493321+0000","last_became_active":"2024-04-27T07:04:15.652383+0000","last_became_peered":"2024-04-27T07:04:15.652383+0000","last_unstale":"2024-04-27T07:04:17.493321+0000","last_undegraded":"2024-04-27T07:04:17.493321+0000","last_fullsized":"2024-04-27T07:04:17.493321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495664+0000","last_change":"2024-04-27T07:04:15.635192+0000","last_active":"2024-04-27T07:04:17.495664+0000","last_peered":"2024-04-27T07:04:17.495664+0000","last_clean":"2024-04-27T07:04:17.495664+0000","last_became_active":"2024-04-27T07:04:15.634845+0000","last_became_peered":"2024-04-27T07:04:15.634845+0000","last_unstale":"2024-04-27T07:04:17.495664+0000","last_undegraded":"2024-04-27T07:04:17.495664+0000","last_fullsized":"2024-04-27T07:04:17.495664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495704+0000","last_change":"2024-04-27T07:04:15.642203+0000","last_active":"2024-04-27T07:04:17.495704+0000","last_peered":"2024-04-27T07:04:17.495704+0000","last_clean":"2024-04-27T07:04:17.495704+0000","last_became_active":"2024-04-27T07:04:15.640450+0000","last_became_peered":"2024-04-27T07:04:15.640450+0000","last_unstale":"2024-04-27T07:04:17.495704+0000","last_undegraded":"2024-04-27T07:04:17.495704+0000","last_fullsized":"2024-04-27T07:04:17.495704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:46.491888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614056+0000","last_change":"2024-04-27T07:04:15.620338+0000","last_active":"2024-04-27T07:04:16.614056+0000","last_peered":"2024-04-27T07:04:16.614056+0000","last_clean":"2024-04-27T07:04:16.614056+0000","last_became_active":"2024-04-27T07:04:15.619521+0000","last_became_peered":"2024-04-27T07:04:15.619521+0000","last_unstale":"2024-04-27T07:04:16.614056+0000","last_undegraded":"2024-04-27T07:04:16.614056+0000","last_fullsized":"2024-04-27T07:04:16.614056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613051+0000","last_change":"2024-04-27T07:04:15.625615+0000","last_active":"2024-04-27T07:04:16.613051+0000","last_peered":"2024-04-27T07:04:16.613051+0000","last_clean":"2024-04-27T07:04:16.613051+0000","last_became_active":"2024-04-27T07:04:15.624734+0000","last_became_peered":"2024-04-27T07:04:15.624734+0000","last_unstale":"2024-04-27T07:04:16.613051+0000","last_undegraded":"2024-04-27T07:04:16.613051+0000","last_fullsized":"2024-04-27T07:04:16.613051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613805+0000","last_change":"2024-04-27T07:04:15.609501+0000","last_active":"2024-04-27T07:04:16.613805+0000","last_peered":"2024-04-27T07:04:16.613805+0000","last_clean":"2024-04-27T07:04:16.613805+0000","last_became_active":"2024-04-27T07:04:15.603709+0000","last_became_peered":"2024-04-27T07:04:15.603709+0000","last_unstale":"2024-04-27T07:04:16.613805+0000","last_undegraded":"2024-04-27T07:04:16.613805+0000","last_fullsized":"2024-04-27T07:04:16.613805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:59.982527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496346+0000","last_change":"2024-04-27T07:04:15.625180+0000","last_active":"2024-04-27T07:04:17.496346+0000","last_peered":"2024-04-27T07:04:17.496346+0000","last_clean":"2024-04-27T07:04:17.496346+0000","last_became_active":"2024-04-27T07:04:15.624748+0000","last_became_peered":"2024-04-27T07:04:15.624748+0000","last_unstale":"2024-04-27T07:04:17.496346+0000","last_undegraded":"2024-04-27T07:04:17.496346+0000","last_fullsized":"2024-04-27T07:04:17.496346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493065+0000","last_change":"2024-04-27T07:04:15.604939+0000","last_active":"2024-04-27T07:04:17.493065+0000","last_peered":"2024-04-27T07:04:17.493065+0000","last_clean":"2024-04-27T07:04:17.493065+0000","last_became_active":"2024-04-27T07:04:15.603580+0000","last_became_peered":"2024-04-27T07:04:15.603580+0000","last_unstale":"2024-04-27T07:04:17.493065+0000","last_undegraded":"2024-04-27T07:04:17.493065+0000","last_fullsized":"2024-04-27T07:04:17.493065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:45.914758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610468+0000","last_change":"2024-04-27T07:04:15.629060+0000","last_active":"2024-04-27T07:04:16.610468+0000","last_peered":"2024-04-27T07:04:16.610468+0000","last_clean":"2024-04-27T07:04:16.610468+0000","last_became_active":"2024-04-27T07:04:15.628737+0000","last_became_peered":"2024-04-27T07:04:15.628737+0000","last_unstale":"2024-04-27T07:04:16.610468+0000","last_undegraded":"2024-04-27T07:04:16.610468+0000","last_fullsized":"2024-04-27T07:04:16.610468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492723+0000","last_change":"2024-04-27T07:04:15.610164+0000","last_active":"2024-04-27T07:04:17.492723+0000","last_peered":"2024-04-27T07:04:17.492723+0000","last_clean":"2024-04-27T07:04:17.492723+0000","last_became_active":"2024-04-27T07:04:15.609290+0000","last_became_peered":"2024-04-27T07:04:15.609290+0000","last_unstale":"2024-04-27T07:04:17.492723+0000","last_undegraded":"2024-04-27T07:04:17.492723+0000","last_fullsized":"2024-04-27T07:04:17.492723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494381+0000","last_change":"2024-04-27T07:04:15.647816+0000","last_active":"2024-04-27T07:04:17.494381+0000","last_peered":"2024-04-27T07:04:17.494381+0000","last_clean":"2024-04-27T07:04:17.494381+0000","last_became_active":"2024-04-27T07:04:15.646892+0000","last_became_peered":"2024-04-27T07:04:15.646892+0000","last_unstale":"2024-04-27T07:04:17.494381+0000","last_undegraded":"2024-04-27T07:04:17.494381+0000","last_fullsized":"2024-04-27T07:04:17.494381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:01.732848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493299+0000","last_change":"2024-04-27T07:04:15.609407+0000","last_active":"2024-04-27T07:04:17.493299+0000","last_peered":"2024-04-27T07:04:17.493299+0000","last_clean":"2024-04-27T07:04:17.493299+0000","last_became_active":"2024-04-27T07:04:15.608565+0000","last_became_peered":"2024-04-27T07:04:15.608565+0000","last_unstale":"2024-04-27T07:04:17.493299+0000","last_undegraded":"2024-04-27T07:04:17.493299+0000","last_fullsized":"2024-04-27T07:04:17.493299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:37.376293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495719+0000","last_change":"2024-04-27T07:04:15.627082+0000","last_active":"2024-04-27T07:04:17.495719+0000","last_peered":"2024-04-27T07:04:17.495719+0000","last_clean":"2024-04-27T07:04:17.495719+0000","last_became_active":"2024-04-27T07:04:15.626302+0000","last_became_peered":"2024-04-27T07:04:15.626302+0000","last_unstale":"2024-04-27T07:04:17.495719+0000","last_undegraded":"2024-04-27T07:04:17.495719+0000","last_fullsized":"2024-04-27T07:04:17.495719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:01.732848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615116+0000","last_change":"2024-04-27T07:04:15.610661+0000","last_active":"2024-04-27T07:04:16.615116+0000","last_peered":"2024-04-27T07:04:16.615116+0000","last_clean":"2024-04-27T07:04:16.615116+0000","last_became_active":"2024-04-27T07:04:15.609899+0000","last_became_peered":"2024-04-27T07:04:15.609899+0000","last_unstale":"2024-04-27T07:04:16.615116+0000","last_undegraded":"2024-04-27T07:04:16.615116+0000","last_fullsized":"2024-04-27T07:04:16.615116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494321+0000","last_change":"2024-04-27T07:04:15.599714+0000","last_active":"2024-04-27T07:04:17.494321+0000","last_peered":"2024-04-27T07:04:17.494321+0000","last_clean":"2024-04-27T07:04:17.494321+0000","last_became_active":"2024-04-27T07:04:15.598855+0000","last_became_peered":"2024-04-27T07:04:15.598855+0000","last_unstale":"2024-04-27T07:04:17.494321+0000","last_undegraded":"2024-04-27T07:04:17.494321+0000","last_fullsized":"2024-04-27T07:04:17.494321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493334+0000","last_change":"2024-04-27T07:04:15.634881+0000","last_active":"2024-04-27T07:04:17.493334+0000","last_peered":"2024-04-27T07:04:17.493334+0000","last_clean":"2024-04-27T07:04:17.493334+0000","last_became_active":"2024-04-27T07:04:15.629428+0000","last_became_peered":"2024-04-27T07:04:15.629428+0000","last_unstale":"2024-04-27T07:04:17.493334+0000","last_undegraded":"2024-04-27T07:04:17.493334+0000","last_fullsized":"2024-04-27T07:04:17.493334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:40:20.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495828+0000","last_change":"2024-04-27T07:04:15.630015+0000","last_active":"2024-04-27T07:04:17.495828+0000","last_peered":"2024-04-27T07:04:17.495828+0000","last_clean":"2024-04-27T07:04:17.495828+0000","last_became_active":"2024-04-27T07:04:15.629177+0000","last_became_peered":"2024-04-27T07:04:15.629177+0000","last_unstale":"2024-04-27T07:04:17.495828+0000","last_undegraded":"2024-04-27T07:04:17.495828+0000","last_fullsized":"2024-04-27T07:04:17.495828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493333+0000","last_change":"2024-04-27T07:04:15.619427+0000","last_active":"2024-04-27T07:04:17.493333+0000","last_peered":"2024-04-27T07:04:17.493333+0000","last_clean":"2024-04-27T07:04:17.493333+0000","last_became_active":"2024-04-27T07:04:15.617987+0000","last_became_peered":"2024-04-27T07:04:15.617987+0000","last_unstale":"2024-04-27T07:04:17.493333+0000","last_undegraded":"2024-04-27T07:04:17.493333+0000","last_fullsized":"2024-04-27T07:04:17.493333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:39:04.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496704+0000","last_change":"2024-04-27T07:04:15.614547+0000","last_active":"2024-04-27T07:04:17.496704+0000","last_peered":"2024-04-27T07:04:17.496704+0000","last_clean":"2024-04-27T07:04:17.496704+0000","last_became_active":"2024-04-27T07:04:15.614210+0000","last_became_peered":"2024-04-27T07:04:15.614210+0000","last_unstale":"2024-04-27T07:04:17.496704+0000","last_undegraded":"2024-04-27T07:04:17.496704+0000","last_fullsized":"2024-04-27T07:04:17.496704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:40:20.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614461+0000","last_change":"2024-04-27T07:04:15.650072+0000","last_active":"2024-04-27T07:04:16.614461+0000","last_peered":"2024-04-27T07:04:16.614461+0000","last_clean":"2024-04-27T07:04:16.614461+0000","last_became_active":"2024-04-27T07:04:15.649461+0000","last_became_peered":"2024-04-27T07:04:15.649461+0000","last_unstale":"2024-04-27T07:04:16.614461+0000","last_undegraded":"2024-04-27T07:04:16.614461+0000","last_fullsized":"2024-04-27T07:04:16.614461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610542+0000","last_change":"2024-04-27T07:04:15.625742+0000","last_active":"2024-04-27T07:04:16.610542+0000","last_peered":"2024-04-27T07:04:16.610542+0000","last_clean":"2024-04-27T07:04:16.610542+0000","last_became_active":"2024-04-27T07:04:15.625049+0000","last_became_peered":"2024-04-27T07:04:15.625049+0000","last_unstale":"2024-04-27T07:04:16.610542+0000","last_undegraded":"2024-04-27T07:04:16.610542+0000","last_fullsized":"2024-04-27T07:04:16.610542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496813+0000","last_change":"2024-04-27T07:04:15.629376+0000","last_active":"2024-04-27T07:04:17.496813+0000","last_peered":"2024-04-27T07:04:17.496813+0000","last_clean":"2024-04-27T07:04:17.496813+0000","last_became_active":"2024-04-27T07:04:15.629058+0000","last_became_peered":"2024-04-27T07:04:15.629058+0000","last_unstale":"2024-04-27T07:04:17.496813+0000","last_undegraded":"2024-04-27T07:04:17.496813+0000","last_fullsized":"2024-04-27T07:04:17.496813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494539+0000","last_change":"2024-04-27T07:04:15.620883+0000","last_active":"2024-04-27T07:04:17.494539+0000","last_peered":"2024-04-27T07:04:17.494539+0000","last_clean":"2024-04-27T07:04:17.494539+0000","last_became_active":"2024-04-27T07:04:15.620549+0000","last_became_peered":"2024-04-27T07:04:15.620549+0000","last_unstale":"2024-04-27T07:04:17.494539+0000","last_undegraded":"2024-04-27T07:04:17.494539+0000","last_fullsized":"2024-04-27T07:04:17.494539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493778+0000","last_change":"2024-04-27T07:04:15.631001+0000","last_active":"2024-04-27T07:04:17.493778+0000","last_peered":"2024-04-27T07:04:17.493778+0000","last_clean":"2024-04-27T07:04:17.493778+0000","last_became_active":"2024-04-27T07:04:15.628448+0000","last_became_peered":"2024-04-27T07:04:15.628448+0000","last_unstale":"2024-04-27T07:04:17.493778+0000","last_undegraded":"2024-04-27T07:04:17.493778+0000","last_fullsized":"2024-04-27T07:04:17.493778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493518+0000","last_change":"2024-04-27T07:04:15.623687+0000","last_active":"2024-04-27T07:04:17.493518+0000","last_peered":"2024-04-27T07:04:17.493518+0000","last_clean":"2024-04-27T07:04:17.493518+0000","last_became_active":"2024-04-27T07:04:15.619092+0000","last_became_peered":"2024-04-27T07:04:15.619092+0000","last_unstale":"2024-04-27T07:04:17.493518+0000","last_undegraded":"2024-04-27T07:04:17.493518+0000","last_fullsized":"2024-04-27T07:04:17.493518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:50.960522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612331+0000","last_change":"2024-04-27T07:04:15.601950+0000","last_active":"2024-04-27T07:04:16.612331+0000","last_peered":"2024-04-27T07:04:16.612331+0000","last_clean":"2024-04-27T07:04:16.612331+0000","last_became_active":"2024-04-27T07:04:15.601144+0000","last_became_peered":"2024-04-27T07:04:15.601144+0000","last_unstale":"2024-04-27T07:04:16.612331+0000","last_undegraded":"2024-04-27T07:04:16.612331+0000","last_fullsized":"2024-04-27T07:04:16.612331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613553+0000","last_change":"2024-04-27T07:04:15.631943+0000","last_active":"2024-04-27T07:04:16.613553+0000","last_peered":"2024-04-27T07:04:16.613553+0000","last_clean":"2024-04-27T07:04:16.613553+0000","last_became_active":"2024-04-27T07:04:15.619207+0000","last_became_peered":"2024-04-27T07:04:15.619207+0000","last_unstale":"2024-04-27T07:04:16.613553+0000","last_undegraded":"2024-04-27T07:04:16.613553+0000","last_fullsized":"2024-04-27T07:04:16.613553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:58.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496502+0000","last_change":"2024-04-27T07:04:14.615664+0000","last_active":"2024-04-27T07:04:17.496502+0000","last_peered":"2024-04-27T07:04:17.496502+0000","last_clean":"2024-04-27T07:04:17.496502+0000","last_became_active":"2024-04-27T07:04:14.615220+0000","last_became_peered":"2024-04-27T07:04:14.615220+0000","last_unstale":"2024-04-27T07:04:17.496502+0000","last_undegraded":"2024-04-27T07:04:17.496502+0000","last_fullsized":"2024-04-27T07:04:17.496502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494747+0000","last_change":"2024-04-27T07:04:15.619876+0000","last_active":"2024-04-27T07:04:17.494747+0000","last_peered":"2024-04-27T07:04:17.494747+0000","last_clean":"2024-04-27T07:04:17.494747+0000","last_became_active":"2024-04-27T07:04:15.619131+0000","last_became_peered":"2024-04-27T07:04:15.619131+0000","last_unstale":"2024-04-27T07:04:17.494747+0000","last_undegraded":"2024-04-27T07:04:17.494747+0000","last_fullsized":"2024-04-27T07:04:17.494747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614112+0000","last_change":"2024-04-27T07:04:14.611792+0000","last_active":"2024-04-27T07:04:16.614112+0000","last_peered":"2024-04-27T07:04:16.614112+0000","last_clean":"2024-04-27T07:04:16.614112+0000","last_became_active":"2024-04-27T07:04:14.609221+0000","last_became_peered":"2024-04-27T07:04:14.609221+0000","last_unstale":"2024-04-27T07:04:16.614112+0000","last_undegraded":"2024-04-27T07:04:16.614112+0000","last_fullsized":"2024-04-27T07:04:16.614112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495969+0000","last_change":"2024-04-27T07:04:14.608168+0000","last_active":"2024-04-27T07:04:17.495969+0000","last_peered":"2024-04-27T07:04:17.495969+0000","last_clean":"2024-04-27T07:04:17.495969+0000","last_became_active":"2024-04-27T07:04:14.607735+0000","last_became_peered":"2024-04-27T07:04:14.607735+0000","last_unstale":"2024-04-27T07:04:17.495969+0000","last_undegraded":"2024-04-27T07:04:17.495969+0000","last_fullsized":"2024-04-27T07:04:17.495969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:06.395968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495767+0000","last_change":"2024-04-27T07:04:15.612016+0000","last_active":"2024-04-27T07:04:17.495767+0000","last_peered":"2024-04-27T07:04:17.495767+0000","last_clean":"2024-04-27T07:04:17.495767+0000","last_became_active":"2024-04-27T07:04:15.611658+0000","last_became_peered":"2024-04-27T07:04:15.611658+0000","last_unstale":"2024-04-27T07:04:17.495767+0000","last_undegraded":"2024-04-27T07:04:17.495767+0000","last_fullsized":"2024-04-27T07:04:17.495767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492906+0000","last_change":"2024-04-27T07:04:14.620476+0000","last_active":"2024-04-27T07:04:17.492906+0000","last_peered":"2024-04-27T07:04:17.492906+0000","last_clean":"2024-04-27T07:04:17.492906+0000","last_became_active":"2024-04-27T07:04:14.617893+0000","last_became_peered":"2024-04-27T07:04:14.617893+0000","last_unstale":"2024-04-27T07:04:17.492906+0000","last_undegraded":"2024-04-27T07:04:17.492906+0000","last_fullsized":"2024-04-27T07:04:17.492906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610222+0000","last_change":"2024-04-27T07:04:15.624591+0000","last_active":"2024-04-27T07:04:16.610222+0000","last_peered":"2024-04-27T07:04:16.610222+0000","last_clean":"2024-04-27T07:04:16.610222+0000","last_became_active":"2024-04-27T07:04:15.624299+0000","last_became_peered":"2024-04-27T07:04:15.624299+0000","last_unstale":"2024-04-27T07:04:16.610222+0000","last_undegraded":"2024-04-27T07:04:16.610222+0000","last_fullsized":"2024-04-27T07:04:16.610222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614771+0000","last_change":"2024-04-27T07:04:14.621378+0000","last_active":"2024-04-27T07:04:16.614771+0000","last_peered":"2024-04-27T07:04:16.614771+0000","last_clean":"2024-04-27T07:04:16.614771+0000","last_became_active":"2024-04-27T07:04:14.620678+0000","last_became_peered":"2024-04-27T07:04:14.620678+0000","last_unstale":"2024-04-27T07:04:16.614771+0000","last_undegraded":"2024-04-27T07:04:16.614771+0000","last_fullsized":"2024-04-27T07:04:16.614771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493724+0000","last_change":"2024-04-27T07:04:15.647396+0000","last_active":"2024-04-27T07:04:17.493724+0000","last_peered":"2024-04-27T07:04:17.493724+0000","last_clean":"2024-04-27T07:04:17.493724+0000","last_became_active":"2024-04-27T07:04:15.646774+0000","last_became_peered":"2024-04-27T07:04:15.646774+0000","last_unstale":"2024-04-27T07:04:17.493724+0000","last_undegraded":"2024-04-27T07:04:17.493724+0000","last_fullsized":"2024-04-27T07:04:17.493724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493229+0000","last_change":"2024-04-27T07:04:14.621935+0000","last_active":"2024-04-27T07:04:17.493229+0000","last_peered":"2024-04-27T07:04:17.493229+0000","last_clean":"2024-04-27T07:04:17.493229+0000","last_became_active":"2024-04-27T07:04:14.620120+0000","last_became_peered":"2024-04-27T07:04:14.620120+0000","last_unstale":"2024-04-27T07:04:17.493229+0000","last_undegraded":"2024-04-27T07:04:17.493229+0000","last_fullsized":"2024-04-27T07:04:17.493229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494443+0000","last_change":"2024-04-27T07:04:15.626451+0000","last_active":"2024-04-27T07:04:17.494443+0000","last_peered":"2024-04-27T07:04:17.494443+0000","last_clean":"2024-04-27T07:04:17.494443+0000","last_became_active":"2024-04-27T07:04:15.626147+0000","last_became_peered":"2024-04-27T07:04:15.626147+0000","last_unstale":"2024-04-27T07:04:17.494443+0000","last_undegraded":"2024-04-27T07:04:17.494443+0000","last_fullsized":"2024-04-27T07:04:17.494443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610433+0000","last_change":"2024-04-27T07:04:14.619618+0000","last_active":"2024-04-27T07:04:16.610433+0000","last_peered":"2024-04-27T07:04:16.610433+0000","last_clean":"2024-04-27T07:04:16.610433+0000","last_became_active":"2024-04-27T07:04:14.619155+0000","last_became_peered":"2024-04-27T07:04:14.619155+0000","last_unstale":"2024-04-27T07:04:16.610433+0000","last_undegraded":"2024-04-27T07:04:16.610433+0000","last_fullsized":"2024-04-27T07:04:16.610433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614234+0000","last_change":"2024-04-27T07:04:15.628583+0000","last_active":"2024-04-27T07:04:16.614234+0000","last_peered":"2024-04-27T07:04:16.614234+0000","last_clean":"2024-04-27T07:04:16.614234+0000","last_became_active":"2024-04-27T07:04:15.621801+0000","last_became_peered":"2024-04-27T07:04:15.621801+0000","last_unstale":"2024-04-27T07:04:16.614234+0000","last_undegraded":"2024-04-27T07:04:16.614234+0000","last_fullsized":"2024-04-27T07:04:16.614234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:37.376293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615199+0000","last_change":"2024-04-27T07:04:14.609138+0000","last_active":"2024-04-27T07:04:16.615199+0000","last_peered":"2024-04-27T07:04:16.615199+0000","last_clean":"2024-04-27T07:04:16.615199+0000","last_became_active":"2024-04-27T07:04:14.608374+0000","last_became_peered":"2024-04-27T07:04:14.608374+0000","last_unstale":"2024-04-27T07:04:16.615199+0000","last_undegraded":"2024-04-27T07:04:16.615199+0000","last_fullsized":"2024-04-27T07:04:16.615199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495184+0000","last_change":"2024-04-27T07:04:15.651834+0000","last_active":"2024-04-27T07:04:17.495184+0000","last_peered":"2024-04-27T07:04:17.495184+0000","last_clean":"2024-04-27T07:04:17.495184+0000","last_became_active":"2024-04-27T07:04:15.651286+0000","last_became_peered":"2024-04-27T07:04:15.651286+0000","last_unstale":"2024-04-27T07:04:17.495184+0000","last_undegraded":"2024-04-27T07:04:17.495184+0000","last_fullsized":"2024-04-27T07:04:17.495184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:00:38.115028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494797+0000","last_change":"2024-04-27T07:04:14.616280+0000","last_active":"2024-04-27T07:04:17.494797+0000","last_peered":"2024-04-27T07:04:17.494797+0000","last_clean":"2024-04-27T07:04:17.494797+0000","last_became_active":"2024-04-27T07:04:14.615920+0000","last_became_peered":"2024-04-27T07:04:14.615920+0000","last_unstale":"2024-04-27T07:04:17.494797+0000","last_undegraded":"2024-04-27T07:04:17.494797+0000","last_fullsized":"2024-04-27T07:04:17.494797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:06.395968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493937+0000","last_change":"2024-04-27T07:04:15.647042+0000","last_active":"2024-04-27T07:04:17.493937+0000","last_peered":"2024-04-27T07:04:17.493937+0000","last_clean":"2024-04-27T07:04:17.493937+0000","last_became_active":"2024-04-27T07:04:15.646683+0000","last_became_peered":"2024-04-27T07:04:15.646683+0000","last_unstale":"2024-04-27T07:04:17.493937+0000","last_undegraded":"2024-04-27T07:04:17.493937+0000","last_fullsized":"2024-04-27T07:04:17.493937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:05:55.398463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493914+0000","last_change":"2024-04-27T07:04:14.620284+0000","last_active":"2024-04-27T07:04:17.493914+0000","last_peered":"2024-04-27T07:04:17.493914+0000","last_clean":"2024-04-27T07:04:17.493914+0000","last_became_active":"2024-04-27T07:04:14.619379+0000","last_became_peered":"2024-04-27T07:04:14.619379+0000","last_unstale":"2024-04-27T07:04:17.493914+0000","last_undegraded":"2024-04-27T07:04:17.493914+0000","last_fullsized":"2024-04-27T07:04:17.493914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495014+0000","last_change":"2024-04-27T07:04:15.635161+0000","last_active":"2024-04-27T07:04:17.495014+0000","last_peered":"2024-04-27T07:04:17.495014+0000","last_clean":"2024-04-27T07:04:17.495014+0000","last_became_active":"2024-04-27T07:04:15.634423+0000","last_became_peered":"2024-04-27T07:04:15.634423+0000","last_unstale":"2024-04-27T07:04:17.495014+0000","last_undegraded":"2024-04-27T07:04:17.495014+0000","last_fullsized":"2024-04-27T07:04:17.495014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:17.226045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612693+0000","last_change":"2024-04-27T07:04:14.612975+0000","last_active":"2024-04-27T07:04:16.612693+0000","last_peered":"2024-04-27T07:04:16.612693+0000","last_clean":"2024-04-27T07:04:16.612693+0000","last_became_active":"2024-04-27T07:04:14.608937+0000","last_became_peered":"2024-04-27T07:04:14.608937+0000","last_unstale":"2024-04-27T07:04:16.612693+0000","last_undegraded":"2024-04-27T07:04:16.612693+0000","last_fullsized":"2024-04-27T07:04:16.612693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612667+0000","last_change":"2024-04-27T07:04:15.601024+0000","last_active":"2024-04-27T07:04:16.612667+0000","last_peered":"2024-04-27T07:04:16.612667+0000","last_clean":"2024-04-27T07:04:16.612667+0000","last_became_active":"2024-04-27T07:04:15.600245+0000","last_became_peered":"2024-04-27T07:04:15.600245+0000","last_unstale":"2024-04-27T07:04:16.612667+0000","last_undegraded":"2024-04-27T07:04:16.612667+0000","last_fullsized":"2024-04-27T07:04:16.612667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492667+0000","last_change":"2024-04-27T07:04:14.594793+0000","last_active":"2024-04-27T07:04:17.492667+0000","last_peered":"2024-04-27T07:04:17.492667+0000","last_clean":"2024-04-27T07:04:17.492667+0000","last_became_active":"2024-04-27T07:04:14.594022+0000","last_became_peered":"2024-04-27T07:04:14.594022+0000","last_unstale":"2024-04-27T07:04:17.492667+0000","last_undegraded":"2024-04-27T07:04:17.492667+0000","last_fullsized":"2024-04-27T07:04:17.492667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:01.729925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493727+0000","last_change":"2024-04-27T07:04:15.639932+0000","last_active":"2024-04-27T07:04:17.493727+0000","last_peered":"2024-04-27T07:04:17.493727+0000","last_clean":"2024-04-27T07:04:17.493727+0000","last_became_active":"2024-04-27T07:04:15.639192+0000","last_became_peered":"2024-04-27T07:04:15.639192+0000","last_unstale":"2024-04-27T07:04:17.493727+0000","last_undegraded":"2024-04-27T07:04:17.493727+0000","last_fullsized":"2024-04-27T07:04:17.493727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614105+0000","last_change":"2024-04-27T07:04:14.605930+0000","last_active":"2024-04-27T07:04:16.614105+0000","last_peered":"2024-04-27T07:04:16.614105+0000","last_clean":"2024-04-27T07:04:16.614105+0000","last_became_active":"2024-04-27T07:04:14.604957+0000","last_became_peered":"2024-04-27T07:04:14.604957+0000","last_unstale":"2024-04-27T07:04:16.614105+0000","last_undegraded":"2024-04-27T07:04:16.614105+0000","last_fullsized":"2024-04-27T07:04:16.614105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:15.465739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495558+0000","last_change":"2024-04-27T07:04:15.635261+0000","last_active":"2024-04-27T07:04:17.495558+0000","last_peered":"2024-04-27T07:04:17.495558+0000","last_clean":"2024-04-27T07:04:17.495558+0000","last_became_active":"2024-04-27T07:04:15.634020+0000","last_became_peered":"2024-04-27T07:04:15.634020+0000","last_unstale":"2024-04-27T07:04:17.495558+0000","last_undegraded":"2024-04-27T07:04:17.495558+0000","last_fullsized":"2024-04-27T07:04:17.495558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:08.890952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493458+0000","last_change":"2024-04-27T07:04:14.600680+0000","last_active":"2024-04-27T07:04:17.493458+0000","last_peered":"2024-04-27T07:04:17.493458+0000","last_clean":"2024-04-27T07:04:17.493458+0000","last_became_active":"2024-04-27T07:04:14.600233+0000","last_became_peered":"2024-04-27T07:04:14.600233+0000","last_unstale":"2024-04-27T07:04:17.493458+0000","last_undegraded":"2024-04-27T07:04:17.493458+0000","last_fullsized":"2024-04-27T07:04:17.493458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495918+0000","last_change":"2024-04-27T07:04:15.643279+0000","last_active":"2024-04-27T07:04:17.495918+0000","last_peered":"2024-04-27T07:04:17.495918+0000","last_clean":"2024-04-27T07:04:17.495918+0000","last_became_active":"2024-04-27T07:04:15.641303+0000","last_became_peered":"2024-04-27T07:04:15.641303+0000","last_unstale":"2024-04-27T07:04:17.495918+0000","last_undegraded":"2024-04-27T07:04:17.495918+0000","last_fullsized":"2024-04-27T07:04:17.495918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:02:46.002992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610599+0000","last_change":"2024-04-27T07:04:14.617808+0000","last_active":"2024-04-27T07:04:16.610599+0000","last_peered":"2024-04-27T07:04:16.610599+0000","last_clean":"2024-04-27T07:04:16.610599+0000","last_became_active":"2024-04-27T07:04:14.617505+0000","last_became_peered":"2024-04-27T07:04:14.617505+0000","last_unstale":"2024-04-27T07:04:16.610599+0000","last_undegraded":"2024-04-27T07:04:16.610599+0000","last_fullsized":"2024-04-27T07:04:16.610599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494318+0000","last_change":"2024-04-27T07:04:15.619586+0000","last_active":"2024-04-27T07:04:17.494318+0000","last_peered":"2024-04-27T07:04:17.494318+0000","last_clean":"2024-04-27T07:04:17.494318+0000","last_became_active":"2024-04-27T07:04:15.617842+0000","last_became_peered":"2024-04-27T07:04:15.617842+0000","last_unstale":"2024-04-27T07:04:17.494318+0000","last_undegraded":"2024-04-27T07:04:17.494318+0000","last_fullsized":"2024-04-27T07:04:17.494318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:46.491888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493601+0000","last_change":"2024-04-27T07:04:14.619149+0000","last_active":"2024-04-27T07:04:17.493601+0000","last_peered":"2024-04-27T07:04:17.493601+0000","last_clean":"2024-04-27T07:04:17.493601+0000","last_became_active":"2024-04-27T07:04:14.618660+0000","last_became_peered":"2024-04-27T07:04:14.618660+0000","last_unstale":"2024-04-27T07:04:17.493601+0000","last_undegraded":"2024-04-27T07:04:17.493601+0000","last_fullsized":"2024-04-27T07:04:17.493601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495600+0000","last_change":"2024-04-27T07:04:15.640980+0000","last_active":"2024-04-27T07:04:17.495600+0000","last_peered":"2024-04-27T07:04:17.495600+0000","last_clean":"2024-04-27T07:04:17.495600+0000","last_became_active":"2024-04-27T07:04:15.640289+0000","last_became_peered":"2024-04-27T07:04:15.640289+0000","last_unstale":"2024-04-27T07:04:17.495600+0000","last_undegraded":"2024-04-27T07:04:17.495600+0000","last_fullsized":"2024-04-27T07:04:17.495600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:19.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492606+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:04:17.492606+0000","last_peered":"2024-04-27T07:04:17.492606+0000","last_clean":"2024-04-27T07:04:17.492606+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:04:17.492606+0000","last_undegraded":"2024-04-27T07:04:17.492606+0000","last_fullsized":"2024-04-27T07:04:17.492606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493168+0000","last_change":"2024-04-27T07:04:14.621914+0000","last_active":"2024-04-27T07:04:17.493168+0000","last_peered":"2024-04-27T07:04:17.493168+0000","last_clean":"2024-04-27T07:04:17.493168+0000","last_became_active":"2024-04-27T07:04:14.620901+0000","last_became_peered":"2024-04-27T07:04:14.620901+0000","last_unstale":"2024-04-27T07:04:17.493168+0000","last_undegraded":"2024-04-27T07:04:17.493168+0000","last_fullsized":"2024-04-27T07:04:17.493168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496303+0000","last_change":"2024-04-27T07:04:15.635276+0000","last_active":"2024-04-27T07:04:17.496303+0000","last_peered":"2024-04-27T07:04:17.496303+0000","last_clean":"2024-04-27T07:04:17.496303+0000","last_became_active":"2024-04-27T07:04:15.634946+0000","last_became_peered":"2024-04-27T07:04:15.634946+0000","last_unstale":"2024-04-27T07:04:17.496303+0000","last_undegraded":"2024-04-27T07:04:17.496303+0000","last_fullsized":"2024-04-27T07:04:17.496303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492322+0000","last_change":"2024-04-27T07:04:14.628608+0000","last_active":"2024-04-27T07:04:17.492322+0000","last_peered":"2024-04-27T07:04:17.492322+0000","last_clean":"2024-04-27T07:04:17.492322+0000","last_became_active":"2024-04-27T07:04:14.626795+0000","last_became_peered":"2024-04-27T07:04:14.626795+0000","last_unstale":"2024-04-27T07:04:17.492322+0000","last_undegraded":"2024-04-27T07:04:17.492322+0000","last_fullsized":"2024-04-27T07:04:17.492322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:32:26.049320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493125+0000","last_change":"2024-04-27T07:04:15.604705+0000","last_active":"2024-04-27T07:04:17.493125+0000","last_peered":"2024-04-27T07:04:17.493125+0000","last_clean":"2024-04-27T07:04:17.493125+0000","last_became_active":"2024-04-27T07:04:15.603226+0000","last_became_peered":"2024-04-27T07:04:15.603226+0000","last_unstale":"2024-04-27T07:04:17.493125+0000","last_undegraded":"2024-04-27T07:04:17.493125+0000","last_fullsized":"2024-04-27T07:04:17.493125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:24.246666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492975+0000","last_change":"2024-04-27T07:04:14.601261+0000","last_active":"2024-04-27T07:04:17.492975+0000","last_peered":"2024-04-27T07:04:17.492975+0000","last_clean":"2024-04-27T07:04:17.492975+0000","last_became_active":"2024-04-27T07:04:14.600555+0000","last_became_peered":"2024-04-27T07:04:14.600555+0000","last_unstale":"2024-04-27T07:04:17.492975+0000","last_undegraded":"2024-04-27T07:04:17.492975+0000","last_fullsized":"2024-04-27T07:04:17.492975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610538+0000","last_change":"2024-04-27T07:04:15.632176+0000","last_active":"2024-04-27T07:04:16.610538+0000","last_peered":"2024-04-27T07:04:16.610538+0000","last_clean":"2024-04-27T07:04:16.610538+0000","last_became_active":"2024-04-27T07:04:15.631861+0000","last_became_peered":"2024-04-27T07:04:15.631861+0000","last_unstale":"2024-04-27T07:04:16.610538+0000","last_undegraded":"2024-04-27T07:04:16.610538+0000","last_fullsized":"2024-04-27T07:04:16.610538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494374+0000","last_change":"2024-04-27T07:04:14.623584+0000","last_active":"2024-04-27T07:04:17.494374+0000","last_peered":"2024-04-27T07:04:17.494374+0000","last_clean":"2024-04-27T07:04:17.494374+0000","last_became_active":"2024-04-27T07:04:14.620709+0000","last_became_peered":"2024-04-27T07:04:14.620709+0000","last_unstale":"2024-04-27T07:04:17.494374+0000","last_undegraded":"2024-04-27T07:04:17.494374+0000","last_fullsized":"2024-04-27T07:04:17.494374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614757+0000","last_change":"2024-04-27T07:04:15.633827+0000","last_active":"2024-04-27T07:04:16.614757+0000","last_peered":"2024-04-27T07:04:16.614757+0000","last_clean":"2024-04-27T07:04:16.614757+0000","last_became_active":"2024-04-27T07:04:15.632912+0000","last_became_peered":"2024-04-27T07:04:15.632912+0000","last_unstale":"2024-04-27T07:04:16.614757+0000","last_undegraded":"2024-04-27T07:04:16.614757+0000","last_fullsized":"2024-04-27T07:04:16.614757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:50.960522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610382+0000","last_change":"2024-04-27T07:04:14.603584+0000","last_active":"2024-04-27T07:04:16.610382+0000","last_peered":"2024-04-27T07:04:16.610382+0000","last_clean":"2024-04-27T07:04:16.610382+0000","last_became_active":"2024-04-27T07:04:14.602186+0000","last_became_peered":"2024-04-27T07:04:14.602186+0000","last_unstale":"2024-04-27T07:04:16.610382+0000","last_undegraded":"2024-04-27T07:04:16.610382+0000","last_fullsized":"2024-04-27T07:04:16.610382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494048+0000","last_change":"2024-04-27T07:04:15.620627+0000","last_active":"2024-04-27T07:04:17.494048+0000","last_peered":"2024-04-27T07:04:17.494048+0000","last_clean":"2024-04-27T07:04:17.494048+0000","last_became_active":"2024-04-27T07:04:15.619725+0000","last_became_peered":"2024-04-27T07:04:15.619725+0000","last_unstale":"2024-04-27T07:04:17.494048+0000","last_undegraded":"2024-04-27T07:04:17.494048+0000","last_fullsized":"2024-04-27T07:04:17.494048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:17.226045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495737+0000","last_change":"2024-04-27T07:04:14.630298+0000","last_active":"2024-04-27T07:04:17.495737+0000","last_peered":"2024-04-27T07:04:17.495737+0000","last_clean":"2024-04-27T07:04:17.495737+0000","last_became_active":"2024-04-27T07:04:14.629539+0000","last_became_peered":"2024-04-27T07:04:14.629539+0000","last_unstale":"2024-04-27T07:04:17.495737+0000","last_undegraded":"2024-04-27T07:04:17.495737+0000","last_fullsized":"2024-04-27T07:04:17.495737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:44.902463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610300+0000","last_change":"2024-04-27T07:04:15.604054+0000","last_active":"2024-04-27T07:04:16.610300+0000","last_peered":"2024-04-27T07:04:16.610300+0000","last_clean":"2024-04-27T07:04:16.610300+0000","last_became_active":"2024-04-27T07:04:15.603243+0000","last_became_peered":"2024-04-27T07:04:15.603243+0000","last_unstale":"2024-04-27T07:04:16.610300+0000","last_undegraded":"2024-04-27T07:04:16.610300+0000","last_fullsized":"2024-04-27T07:04:16.610300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610793+0000","last_change":"2024-04-27T07:04:14.614799+0000","last_active":"2024-04-27T07:04:16.610793+0000","last_peered":"2024-04-27T07:04:16.610793+0000","last_clean":"2024-04-27T07:04:16.610793+0000","last_became_active":"2024-04-27T07:04:14.614348+0000","last_became_peered":"2024-04-27T07:04:14.614348+0000","last_unstale":"2024-04-27T07:04:16.610793+0000","last_undegraded":"2024-04-27T07:04:16.610793+0000","last_fullsized":"2024-04-27T07:04:16.610793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:49.948227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.497067+0000","last_change":"2024-04-27T07:04:15.649374+0000","last_active":"2024-04-27T07:04:17.497067+0000","last_peered":"2024-04-27T07:04:17.497067+0000","last_clean":"2024-04-27T07:04:17.497067+0000","last_became_active":"2024-04-27T07:04:15.649079+0000","last_became_peered":"2024-04-27T07:04:15.649079+0000","last_unstale":"2024-04-27T07:04:17.497067+0000","last_undegraded":"2024-04-27T07:04:17.497067+0000","last_fullsized":"2024-04-27T07:04:17.497067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615045+0000","last_change":"2024-04-27T07:04:14.630929+0000","last_active":"2024-04-27T07:04:16.615045+0000","last_peered":"2024-04-27T07:04:16.615045+0000","last_clean":"2024-04-27T07:04:16.615045+0000","last_became_active":"2024-04-27T07:04:14.630345+0000","last_became_peered":"2024-04-27T07:04:14.630345+0000","last_unstale":"2024-04-27T07:04:16.615045+0000","last_undegraded":"2024-04-27T07:04:16.615045+0000","last_fullsized":"2024-04-27T07:04:16.615045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494516+0000","last_change":"2024-04-27T07:04:15.605948+0000","last_active":"2024-04-27T07:04:17.494516+0000","last_peered":"2024-04-27T07:04:17.494516+0000","last_clean":"2024-04-27T07:04:17.494516+0000","last_became_active":"2024-04-27T07:04:15.603763+0000","last_became_peered":"2024-04-27T07:04:15.603763+0000","last_unstale":"2024-04-27T07:04:17.494516+0000","last_undegraded":"2024-04-27T07:04:17.494516+0000","last_fullsized":"2024-04-27T07:04:17.494516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492302+0000","last_change":"2024-04-27T07:04:14.628661+0000","last_active":"2024-04-27T07:04:17.492302+0000","last_peered":"2024-04-27T07:04:17.492302+0000","last_clean":"2024-04-27T07:04:17.492302+0000","last_became_active":"2024-04-27T07:04:14.626731+0000","last_became_peered":"2024-04-27T07:04:14.626731+0000","last_unstale":"2024-04-27T07:04:17.492302+0000","last_undegraded":"2024-04-27T07:04:17.492302+0000","last_fullsized":"2024-04-27T07:04:17.492302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613464+0000","last_change":"2024-04-27T07:04:15.608033+0000","last_active":"2024-04-27T07:04:16.613464+0000","last_peered":"2024-04-27T07:04:16.613464+0000","last_clean":"2024-04-27T07:04:16.613464+0000","last_became_active":"2024-04-27T07:04:15.607219+0000","last_became_peered":"2024-04-27T07:04:15.607219+0000","last_unstale":"2024-04-27T07:04:16.613464+0000","last_undegraded":"2024-04-27T07:04:16.613464+0000","last_fullsized":"2024-04-27T07:04:16.613464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:02.140237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610281+0000","last_change":"2024-04-27T07:04:14.613406+0000","last_active":"2024-04-27T07:04:16.610281+0000","last_peered":"2024-04-27T07:04:16.610281+0000","last_clean":"2024-04-27T07:04:16.610281+0000","last_became_active":"2024-04-27T07:04:14.613017+0000","last_became_peered":"2024-04-27T07:04:14.613017+0000","last_unstale":"2024-04-27T07:04:16.610281+0000","last_undegraded":"2024-04-27T07:04:16.610281+0000","last_fullsized":"2024-04-27T07:04:16.610281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496138+0000","last_change":"2024-04-27T07:04:15.617597+0000","last_active":"2024-04-27T07:04:17.496138+0000","last_peered":"2024-04-27T07:04:17.496138+0000","last_clean":"2024-04-27T07:04:17.496138+0000","last_became_active":"2024-04-27T07:04:15.616122+0000","last_became_peered":"2024-04-27T07:04:15.616122+0000","last_unstale":"2024-04-27T07:04:17.496138+0000","last_undegraded":"2024-04-27T07:04:17.496138+0000","last_fullsized":"2024-04-27T07:04:17.496138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610542+0000","last_change":"2024-04-27T07:04:14.604022+0000","last_active":"2024-04-27T07:04:16.610542+0000","last_peered":"2024-04-27T07:04:16.610542+0000","last_clean":"2024-04-27T07:04:16.610542+0000","last_became_active":"2024-04-27T07:04:14.603308+0000","last_became_peered":"2024-04-27T07:04:14.603308+0000","last_unstale":"2024-04-27T07:04:16.610542+0000","last_undegraded":"2024-04-27T07:04:16.610542+0000","last_fullsized":"2024-04-27T07:04:16.610542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496561+0000","last_change":"2024-04-27T07:04:15.620378+0000","last_active":"2024-04-27T07:04:17.496561+0000","last_peered":"2024-04-27T07:04:17.496561+0000","last_clean":"2024-04-27T07:04:17.496561+0000","last_became_active":"2024-04-27T07:04:15.620100+0000","last_became_peered":"2024-04-27T07:04:15.620100+0000","last_unstale":"2024-04-27T07:04:17.496561+0000","last_undegraded":"2024-04-27T07:04:17.496561+0000","last_fullsized":"2024-04-27T07:04:17.496561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494886+0000","last_change":"2024-04-27T07:04:14.629851+0000","last_active":"2024-04-27T07:04:17.494886+0000","last_peered":"2024-04-27T07:04:17.494886+0000","last_clean":"2024-04-27T07:04:17.494886+0000","last_became_active":"2024-04-27T07:04:14.629063+0000","last_became_peered":"2024-04-27T07:04:14.629063+0000","last_unstale":"2024-04-27T07:04:17.494886+0000","last_undegraded":"2024-04-27T07:04:17.494886+0000","last_fullsized":"2024-04-27T07:04:17.494886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:49.948227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613962+0000","last_change":"2024-04-27T07:04:15.616289+0000","last_active":"2024-04-27T07:04:16.613962+0000","last_peered":"2024-04-27T07:04:16.613962+0000","last_clean":"2024-04-27T07:04:16.613962+0000","last_became_active":"2024-04-27T07:04:15.615386+0000","last_became_peered":"2024-04-27T07:04:15.615386+0000","last_unstale":"2024-04-27T07:04:16.613962+0000","last_undegraded":"2024-04-27T07:04:16.613962+0000","last_fullsized":"2024-04-27T07:04:16.613962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493841+0000","last_change":"2024-04-27T07:04:15.646050+0000","last_active":"2024-04-27T07:04:17.493841+0000","last_peered":"2024-04-27T07:04:17.493841+0000","last_clean":"2024-04-27T07:04:17.493841+0000","last_became_active":"2024-04-27T07:04:15.645284+0000","last_became_peered":"2024-04-27T07:04:15.645284+0000","last_unstale":"2024-04-27T07:04:17.493841+0000","last_undegraded":"2024-04-27T07:04:17.493841+0000","last_fullsized":"2024-04-27T07:04:17.493841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:11.386203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494836+0000","last_change":"2024-04-27T07:04:14.629860+0000","last_active":"2024-04-27T07:04:17.494836+0000","last_peered":"2024-04-27T07:04:17.494836+0000","last_clean":"2024-04-27T07:04:17.494836+0000","last_became_active":"2024-04-27T07:04:14.629138+0000","last_became_peered":"2024-04-27T07:04:14.629138+0000","last_unstale":"2024-04-27T07:04:17.494836+0000","last_undegraded":"2024-04-27T07:04:17.494836+0000","last_fullsized":"2024-04-27T07:04:17.494836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494061+0000","last_change":"2024-04-27T07:04:15.638833+0000","last_active":"2024-04-27T07:04:17.494061+0000","last_peered":"2024-04-27T07:04:17.494061+0000","last_clean":"2024-04-27T07:04:17.494061+0000","last_became_active":"2024-04-27T07:04:15.638192+0000","last_became_peered":"2024-04-27T07:04:15.638192+0000","last_unstale":"2024-04-27T07:04:17.494061+0000","last_undegraded":"2024-04-27T07:04:17.494061+0000","last_fullsized":"2024-04-27T07:04:17.494061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:00:38.115028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495394+0000","last_change":"2024-04-27T07:04:14.632528+0000","last_active":"2024-04-27T07:04:17.495394+0000","last_peered":"2024-04-27T07:04:17.495394+0000","last_clean":"2024-04-27T07:04:17.495394+0000","last_became_active":"2024-04-27T07:04:14.631089+0000","last_became_peered":"2024-04-27T07:04:14.631089+0000","last_unstale":"2024-04-27T07:04:17.495394+0000","last_undegraded":"2024-04-27T07:04:17.495394+0000","last_fullsized":"2024-04-27T07:04:17.495394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:03:50.734904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494459+0000","last_change":"2024-04-27T07:04:15.626754+0000","last_active":"2024-04-27T07:04:17.494459+0000","last_peered":"2024-04-27T07:04:17.494459+0000","last_clean":"2024-04-27T07:04:17.494459+0000","last_became_active":"2024-04-27T07:04:15.626499+0000","last_became_peered":"2024-04-27T07:04:15.626499+0000","last_unstale":"2024-04-27T07:04:17.494459+0000","last_undegraded":"2024-04-27T07:04:17.494459+0000","last_fullsized":"2024-04-27T07:04:17.494459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496732+0000","last_change":"2024-04-27T07:04:14.598918+0000","last_active":"2024-04-27T07:04:17.496732+0000","last_peered":"2024-04-27T07:04:17.496732+0000","last_clean":"2024-04-27T07:04:17.496732+0000","last_became_active":"2024-04-27T07:04:14.598504+0000","last_became_peered":"2024-04-27T07:04:14.598504+0000","last_unstale":"2024-04-27T07:04:17.496732+0000","last_undegraded":"2024-04-27T07:04:17.496732+0000","last_fullsized":"2024-04-27T07:04:17.496732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:01.729925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496574+0000","last_change":"2024-04-27T07:04:15.625377+0000","last_active":"2024-04-27T07:04:17.496574+0000","last_peered":"2024-04-27T07:04:17.496574+0000","last_clean":"2024-04-27T07:04:17.496574+0000","last_became_active":"2024-04-27T07:04:15.625075+0000","last_became_peered":"2024-04-27T07:04:15.625075+0000","last_unstale":"2024-04-27T07:04:17.496574+0000","last_undegraded":"2024-04-27T07:04:17.496574+0000","last_fullsized":"2024-04-27T07:04:17.496574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:39:04.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494529+0000","last_change":"2024-04-27T07:04:14.626854+0000","last_active":"2024-04-27T07:04:17.494529+0000","last_peered":"2024-04-27T07:04:17.494529+0000","last_clean":"2024-04-27T07:04:17.494529+0000","last_became_active":"2024-04-27T07:04:14.626588+0000","last_became_peered":"2024-04-27T07:04:14.626588+0000","last_unstale":"2024-04-27T07:04:17.494529+0000","last_undegraded":"2024-04-27T07:04:17.494529+0000","last_fullsized":"2024-04-27T07:04:17.494529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:21.930932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:04:24.506 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-27T07:04:24.506 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-27T07:04:24.506 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-27T07:04:24.506 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-27T07:04:24.507 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-27T07:04:24.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.661+0000 7f8ec9c22640 1 Processor -- start 2024-04-27T07:04:24.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec9c22640 1 -- start start 2024-04-27T07:04:24.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec9c22640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8ec40b3db0 0x7f8ec406d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec9c22640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec406dcb0 con 0x7f8ec40b4b70 2024-04-27T07:04:24.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec9c22640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec41448e0 con 0x7f8ec406d840 2024-04-27T07:04:24.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec9c22640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec4144ef0 con 0x7f8ec40b3db0 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec37fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8ec40b4b70 0x7f8ec40b4f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40814/0 (socket says 172.21.15.12:40814) 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec37fe640 1 -- 172.21.15.12:0/587316253 learned_addr learned my addr 172.21.15.12:0/587316253 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1159532969 0 0) 0x7f8ec406dcb0 con 0x7f8ec40b4b70 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec2ffd640 1 --2- 172.21.15.12:0/587316253 >> v2:172.21.15.53:3300/0 conn(0x7f8ec40b3db0 0x7f8ec406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8eb0003530 con 0x7f8ec40b4b70 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1270469934 0 0) 0x7f8eb0003530 con 0x7f8ec40b4b70 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 >> v2:172.21.15.53:3300/0 conn(0x7f8ec40b3db0 msgr2=0x7f8ec406d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 --2- 172.21.15.12:0/587316253 >> v2:172.21.15.53:3300/0 conn(0x7f8ec40b3db0 0x7f8ec406d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 >> v1:172.21.15.53:6789/0 conn(0x7f8ec406d840 legacy=0x7f8ec4143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec4144b80 con 0x7f8ec40b4b70 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f8eb4001a00 con 0x7f8ec40b4b70 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 >> v1:172.21.15.12:6789/0 conn(0x7f8ec40b4b70 legacy=0x7f8ec40b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 --2- 172.21.15.12:0/587316253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8eb0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 --2- 172.21.15.12:0/587316253 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8eb0004810 0x7f8eb0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 --2- 172.21.15.12:0/587316253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00052f0 0x7f8eb000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb000c1c0 con 0x7f8eb00039c0 2024-04-27T07:04:24.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb000c370 con 0x7f8eb00052f0 2024-04-27T07:04:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb000c570 con 0x7f8eb0004810 2024-04-27T07:04:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec37fe640 1 --2- 172.21.15.12:0/587316253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8eb0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec3fff640 1 --2- 172.21.15.12:0/587316253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00052f0 0x7f8eb000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec37fe640 1 -- 172.21.15.12:0/587316253 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8eb0004810 msgr2=0x7f8eb0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec37fe640 1 --2- 172.21.15.12:0/587316253 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8eb0004810 0x7f8eb0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec37fe640 1 -- 172.21.15.12:0/587316253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00052f0 msgr2=0x7f8eb000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec37fe640 1 --2- 172.21.15.12:0/587316253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb00052f0 0x7f8eb000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec37fe640 1 -- 172.21.15.12:0/587316253 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb0003530 con 0x7f8eb00039c0 2024-04-27T07:04:24.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.665+0000 7f8ec3fff640 1 -- 172.21.15.12:0/587316253 reap_dead start 2024-04-27T07:04:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec37fe640 1 --2- 172.21.15.12:0/587316253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8eb0003e00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400a810 tx=0x7f8eb401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985ffce02a28623d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb400a460 con 0x7f8eb00039c0 2024-04-27T07:04:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb40ad650 con 0x7f8eb00039c0 2024-04-27T07:04:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec27fc640 1 -- 172.21.15.12:0/587316253 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40ada80 con 0x7f8eb00039c0 2024-04-27T07:04:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- 172.21.15.12:0/587316253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 msgr2=0x7f8eb0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 --2- 172.21.15.12:0/587316253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8eb0003e00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400a810 tx=0x7f8eb401d340 comp rx=0 tx=0).stop 2024-04-27T07:04:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- 172.21.15.12:0/587316253 shutdown_connections 2024-04-27T07:04:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 --2- 172.21.15.12:0/587316253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8eb0003e00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- 172.21.15.12:0/587316253 >> 172.21.15.12:0/587316253 conn(0x7f8ec4067d10 msgr2=0x7f8ec4068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:24.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- 172.21.15.12:0/587316253 shutdown_connections 2024-04-27T07:04:24.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- 172.21.15.12:0/587316253 wait complete. 2024-04-27T07:04:24.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 Processor -- start 2024-04-27T07:04:24.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- start start 2024-04-27T07:04:24.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8ec40b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ec40b3480 0x7f8ec41e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ec41d0f80 0x7f8ec4072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec4143900 con 0x7f8eb00039c0 2024-04-27T07:04:24.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec4143780 con 0x7f8ec40b3480 2024-04-27T07:04:24.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec4143a80 con 0x7f8ec41d0f80 2024-04-27T07:04:24.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8ec40b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8ec40b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42034/0 (socket says 172.21.15.12:42034) 2024-04-27T07:04:24.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec37fe640 1 -- 172.21.15.12:0/3914517655 learned_addr learned my addr 172.21.15.12:0/3914517655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:24.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec2ffd640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ec40b3480 0x7f8ec41e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec3fff640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ec41d0f80 0x7f8ec4072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec37fe640 1 -- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ec41d0f80 msgr2=0x7f8ec4072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec37fe640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ec41d0f80 0x7f8ec4072aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec37fe640 1 -- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ec40b3480 msgr2=0x7f8ec41e0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec37fe640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ec40b3480 0x7f8ec41e0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec37fe640 1 -- 172.21.15.12:0/3914517655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec40b2740 con 0x7f8eb00039c0 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec37fe640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8ec40b1ae0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4001d00 tx=0x7f8eb0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ebbfff640 1 -- 172.21.15.12:0/3914517655 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40ad430 con 0x7f8eb00039c0 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ebbfff640 1 -- 172.21.15.12:0/3914517655 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb40017c0 con 0x7f8eb00039c0 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- 172.21.15.12:0/3914517655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec40b2ee0 con 0x7f8eb00039c0 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ebbfff640 1 -- 172.21.15.12:0/3914517655 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40ad800 con 0x7f8eb00039c0 2024-04-27T07:04:24.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.669+0000 7f8ec9c22640 1 -- 172.21.15.12:0/3914517655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec4072fe0 con 0x7f8eb00039c0 2024-04-27T07:04:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.673+0000 7f8ec9c22640 1 -- 172.21.15.12:0/3914517655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec40b2940 con 0x7f8eb00039c0 2024-04-27T07:04:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.673+0000 7f8ebbfff640 1 -- 172.21.15.12:0/3914517655 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8eb40ad9c0 con 0x7f8eb00039c0 2024-04-27T07:04:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.673+0000 7f8ebbfff640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ea4068380 0x7f8ea406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.673+0000 7f8ebbfff640 1 -- 172.21.15.12:0/3914517655 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8eb4120bd0 con 0x7f8eb00039c0 2024-04-27T07:04:24.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.673+0000 7f8ec2ffd640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ea4068380 0x7f8ea406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:24.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.673+0000 7f8ec2ffd640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ea4068380 0x7f8ea406a840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006aa0 tx=0x7f8eac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:24.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.677+0000 7f8ebbfff640 1 -- 172.21.15.12:0/3914517655 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8eb40bc060 con 0x7f8eb00039c0 2024-04-27T07:04:24.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.881+0000 7f8ec9c22640 1 -- 172.21.15.12:0/3914517655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8ec4068c20 con 0x7f8eb00039c0 2024-04-27T07:04:24.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.885+0000 7f8ebbfff640 1 -- 172.21.15.12:0/3914517655 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+168081 (secure 0 0 0) 0x7f8eb4017050 con 0x7f8eb00039c0 2024-04-27T07:04:24.893 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 -- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ea4068380 msgr2=0x7f8ea406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ea4068380 0x7f8ea406a840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006aa0 tx=0x7f8eac008040 comp rx=0 tx=0).stop 2024-04-27T07:04:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 -- 172.21.15.12:0/3914517655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 msgr2=0x7f8ec40b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8ec40b1ae0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4001d00 tx=0x7f8eb0005840 comp rx=0 tx=0).stop 2024-04-27T07:04:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 -- 172.21.15.12:0/3914517655 shutdown_connections 2024-04-27T07:04:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ea4068380 0x7f8ea406a840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ec41d0f80 0x7f8ec4072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ec40b3480 0x7f8ec41e0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 --2- 172.21.15.12:0/3914517655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8eb00039c0 0x7f8ec40b1ae0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 -- 172.21.15.12:0/3914517655 >> 172.21.15.12:0/3914517655 conn(0x7f8ec4067d10 msgr2=0x7f8ec409dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 -- 172.21.15.12:0/3914517655 shutdown_connections 2024-04-27T07:04:24.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:24.889+0000 7f8ec9c22640 1 -- 172.21.15.12:0/3914517655 wait complete. 2024-04-27T07:04:24.912 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":452515146},{"type":"v1","addr":"172.21.15.53:6833","nonce":452515146}]},"active_addr":"172.21.15.53:6833/452515146","active_change":"2024-04-27T07:04:02.595631+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":1506956706}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":1570623673}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":863906995}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":1580534915}]}]} 2024-04-27T07:04:24.916 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-27T07:04:24.916 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-27T07:04:24.916 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:25.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f201e189640 1 Processor -- start 2024-04-27T07:04:25.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f201e189640 1 -- start start 2024-04-27T07:04:25.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f201e189640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20181515e0 0x7f20181539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f201e189640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20180b16f0 con 0x7f201806bde0 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f201e189640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f201806c1c0 con 0x7f20180b1290 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f201e189640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f20180b6e60 con 0x7f20181515e0 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20177fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f201806bde0 0x7f20180735e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40820/0 (socket says 172.21.15.12:40820) 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20177fe640 1 -- 172.21.15.12:0/1407492249 learned_addr learned my addr 172.21.15.12:0/1407492249 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f2017fff640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20181515e0 0x7f20181539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52326356 0 0) 0x7f20180b16f0 con 0x7f201806bde0 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2004003530 con 0x7f201806bde0 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1376284133 0 0) 0x7f2004003530 con 0x7f201806bde0 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20181515e0 msgr2=0x7f20181539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20181515e0 0x7f20181539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 >> v1:172.21.15.53:6789/0 conn(0x7f20180b1290 legacy=0x7f201814b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20180afa90 con 0x7f201806bde0 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f2000001680 con 0x7f201806bde0 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 >> v1:172.21.15.12:6789/0 conn(0x7f201806bde0 legacy=0x7f20180735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20181515e0 0x7f20040039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2004003f70 0x7f2004004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004004d50 0x7f20040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f200400bb60 con 0x7f2004003f70 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f200400bce0 con 0x7f2004004d50 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f200400bee0 con 0x7f20181515e0 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f2016ffd640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2004003f70 0x7f2004004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f20177fe640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20181515e0 0x7f20040039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f2016ffd640 1 -- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20181515e0 msgr2=0x7f20040039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f2017fff640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004004d50 0x7f20040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f2016ffd640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20181515e0 0x7f20040039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f2016ffd640 1 -- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004004d50 msgr2=0x7f20040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f2016ffd640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004004d50 0x7f20040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f2016ffd640 1 -- 172.21.15.12:0/1407492249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2004003530 con 0x7f2004003f70 2024-04-27T07:04:25.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.037+0000 7f2017fff640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004004d50 0x7f20040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:25.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f2016ffd640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2004003f70 0x7f2004004810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2008001780 tx=0x7f200801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6352f829aafaa69f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20080013a0 con 0x7f2004003f70 2024-04-27T07:04:25.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20080052f0 con 0x7f2004003f70 2024-04-27T07:04:25.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f20167fc640 1 -- 172.21.15.12:0/1407492249 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008003840 con 0x7f2004003f70 2024-04-27T07:04:25.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f201e189640 1 -- 172.21.15.12:0/1407492249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2004003f70 msgr2=0x7f2004004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f201e189640 1 --2- 172.21.15.12:0/1407492249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2004003f70 0x7f2004004810 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2008001780 tx=0x7f200801c420 comp rx=0 tx=0).stop 2024-04-27T07:04:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f2017fff640 1 -- 172.21.15.12:0/1407492249 reap_dead start 2024-04-27T07:04:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f201e189640 1 -- 172.21.15.12:0/1407492249 shutdown_connections 2024-04-27T07:04:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f201e189640 1 -- 172.21.15.12:0/1407492249 >> 172.21.15.12:0/1407492249 conn(0x7f20180b6ff0 msgr2=0x7f20180b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f201e189640 1 -- 172.21.15.12:0/1407492249 shutdown_connections 2024-04-27T07:04:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f201e189640 1 -- 172.21.15.12:0/1407492249 wait complete. 2024-04-27T07:04:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f201e189640 1 Processor -- start 2024-04-27T07:04:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.041+0000 7f201e189640 1 -- start start 2024-04-27T07:04:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f201e189640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004003f70 0x7f2018148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f201e189640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20180b33a0 0x7f20180b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f201e189640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20180b29a0 0x7f20181e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f201e189640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f201814b960 con 0x7f20180b33a0 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f201e189640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20180af6d0 con 0x7f2004003f70 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f201e189640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f20180b1a10 con 0x7f20180b29a0 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2016ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20180b33a0 0x7f20180b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2016ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20180b33a0 0x7f20180b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42060/0 (socket says 172.21.15.12:42060) 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2016ffd640 1 -- 172.21.15.12:0/2690360880 learned_addr learned my addr 172.21.15.12:0/2690360880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2017fff640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20180b29a0 0x7f20181e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2016ffd640 1 -- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20180b29a0 msgr2=0x7f20181e37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2016ffd640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20180b29a0 0x7f20181e37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f20177fe640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004003f70 0x7f2018148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2016ffd640 1 -- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004003f70 msgr2=0x7f2018148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2016ffd640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004003f70 0x7f2018148620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2016ffd640 1 -- 172.21.15.12:0/2690360880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20180b3b90 con 0x7f20180b33a0 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2017fff640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20180b29a0 0x7f20181e37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f2016ffd640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20180b33a0 0x7f20180b2460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2008005040 tx=0x7f2004005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f200ffff640 1 -- 172.21.15.12:0/2690360880 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20080b9020 con 0x7f20180b33a0 2024-04-27T07:04:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f200ffff640 1 -- 172.21.15.12:0/2690360880 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20080054a0 con 0x7f20180b33a0 2024-04-27T07:04:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f201e189640 1 -- 172.21.15.12:0/2690360880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201809feb0 con 0x7f20180b33a0 2024-04-27T07:04:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f200ffff640 1 -- 172.21.15.12:0/2690360880 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008007a40 con 0x7f20180b33a0 2024-04-27T07:04:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f201e189640 1 -- 172.21.15.12:0/2690360880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20180b3820 con 0x7f20180b33a0 2024-04-27T07:04:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f201e189640 1 -- 172.21.15.12:0/2690360880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2018068c20 con 0x7f20180b33a0 2024-04-27T07:04:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f200ffff640 1 -- 172.21.15.12:0/2690360880 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f20080bc020 con 0x7f20180b33a0 2024-04-27T07:04:25.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f200ffff640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ff80683b0 0x7f1ff806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f200ffff640 1 -- 172.21.15.12:0/2690360880 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2008120ae0 con 0x7f20180b33a0 2024-04-27T07:04:25.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.045+0000 7f20177fe640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ff80683b0 0x7f1ff806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.049+0000 7f20177fe640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ff80683b0 0x7f1ff806a870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2000001060 tx=0x7f2000001120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.049+0000 7f200ffff640 1 -- 172.21.15.12:0/2690360880 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f20080c3030 con 0x7f20180b33a0 2024-04-27T07:04:25.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.213+0000 7f201e189640 1 -- 172.21.15.12:0/2690360880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f201809b160 con 0x7f20180b33a0 2024-04-27T07:04:25.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.217+0000 7f200ffff640 1 -- 172.21.15.12:0/2690360880 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f2008016130 con 0x7f20180b33a0 2024-04-27T07:04:25.222 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:25.222 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:25.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 -- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ff80683b0 msgr2=0x7f1ff806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ff80683b0 0x7f1ff806a870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2000001060 tx=0x7f2000001120 comp rx=0 tx=0).stop 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 -- 172.21.15.12:0/2690360880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20180b33a0 msgr2=0x7f20180b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20180b33a0 0x7f20180b2460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2008005040 tx=0x7f2004005840 comp rx=0 tx=0).stop 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 -- 172.21.15.12:0/2690360880 shutdown_connections 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ff80683b0 0x7f1ff806a870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20180b29a0 0x7f20181e37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20180b33a0 0x7f20180b2460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 --2- 172.21.15.12:0/2690360880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004003f70 0x7f2018148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 -- 172.21.15.12:0/2690360880 >> 172.21.15.12:0/2690360880 conn(0x7f20180b6ff0 msgr2=0x7f20180b5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 -- 172.21.15.12:0/2690360880 shutdown_connections 2024-04-27T07:04:25.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.221+0000 7f201e189640 1 -- 172.21.15.12:0/2690360880 wait complete. 2024-04-27T07:04:25.241 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-27T07:04:25.241 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:25.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c50c45640 1 Processor -- start 2024-04-27T07:04:25.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c50c45640 1 -- start start 2024-04-27T07:04:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c50c45640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4c4c1464c0 0x7f4c4c0b5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c50c45640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c4c0b4920 con 0x7f4c4c13f530 2024-04-27T07:04:25.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c50c45640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c4c0b3d70 con 0x7f4c4c0b5330 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c50c45640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f4c4c0b7bd0 con 0x7f4c4c1464c0 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4ad9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4c4c13f530 0x7f4c4c13f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40830/0 (socket says 172.21.15.12:40830) 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4ad9d640 1 -- 172.21.15.12:0/2679386771 learned_addr learned my addr 172.21.15.12:0/2679386771 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c49d9b640 1 --2- 172.21.15.12:0/2679386771 >> v2:172.21.15.53:3300/0 conn(0x7f4c4c1464c0 0x7f4c4c0b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 274575149 0 0) 0x7f4c4c0b4920 con 0x7f4c4c13f530 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c34003530 con 0x7f4c4c13f530 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 426605902 0 0) 0x7f4c34003530 con 0x7f4c4c13f530 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 >> v2:172.21.15.53:3300/0 conn(0x7f4c4c1464c0 msgr2=0x7f4c4c0b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 --2- 172.21.15.12:0/2679386771 >> v2:172.21.15.53:3300/0 conn(0x7f4c4c1464c0 0x7f4c4c0b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 >> v1:172.21.15.53:6789/0 conn(0x7f4c4c0b5330 legacy=0x7f4c4c145db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c0b2a60 con 0x7f4c4c13f530 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f4c44001a00 con 0x7f4c4c13f530 2024-04-27T07:04:25.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 >> v1:172.21.15.12:6789/0 conn(0x7f4c4c13f530 legacy=0x7f4c4c13f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c340039c0 0x7f4c34003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34004810 0x7f4c34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c340052f0 0x7f4c3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c0b7bd0 con 0x7f4c34004810 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3400c1c0 con 0x7f4c340052f0 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3400c3c0 con 0x7f4c340039c0 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c49d9b640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34004810 0x7f4c34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4ad9d640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c340052f0 0x7f4c3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4a59c640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c340039c0 0x7f4c34003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c49d9b640 1 -- 172.21.15.12:0/2679386771 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c340039c0 msgr2=0x7f4c34003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c49d9b640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c340039c0 0x7f4c34003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c49d9b640 1 -- 172.21.15.12:0/2679386771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c340052f0 msgr2=0x7f4c3400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c49d9b640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c340052f0 0x7f4c3400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c4ad9d640 1 -- 172.21.15.12:0/2679386771 reap_dead start 2024-04-27T07:04:25.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.361+0000 7f4c49d9b640 1 -- 172.21.15.12:0/2679386771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c34003530 con 0x7f4c34004810 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c49d9b640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34004810 0x7f4c34004c90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009ab0 tx=0x7f4c4001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e41510b8b5ba6b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c400b2070 con 0x7f4c34004810 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c400a5530 con 0x7f4c34004810 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c4959a640 1 -- 172.21.15.12:0/2679386771 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c400096b0 con 0x7f4c34004810 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 -- 172.21.15.12:0/2679386771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34004810 msgr2=0x7f4c34004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34004810 0x7f4c34004c90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009ab0 tx=0x7f4c4001ca50 comp rx=0 tx=0).stop 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 -- 172.21.15.12:0/2679386771 shutdown_connections 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 --2- 172.21.15.12:0/2679386771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34004810 0x7f4c34004c90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 -- 172.21.15.12:0/2679386771 >> 172.21.15.12:0/2679386771 conn(0x7f4c4c0bb560 msgr2=0x7f4c4c0bb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 -- 172.21.15.12:0/2679386771 shutdown_connections 2024-04-27T07:04:25.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 -- 172.21.15.12:0/2679386771 wait complete. 2024-04-27T07:04:25.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 Processor -- start 2024-04-27T07:04:25.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 -- start start 2024-04-27T07:04:25.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c34004810 0x7f4c4c0bac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c140720 0x7f4c4c1deef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c1df430 0x7f4c4c1386c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c13fcd0 con 0x7f4c4c1df430 2024-04-27T07:04:25.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c13fb50 con 0x7f4c4c140720 2024-04-27T07:04:25.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c50c45640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f4c4c13fe50 con 0x7f4c34004810 2024-04-27T07:04:25.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c4ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c1df430 0x7f4c4c1386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c4ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c1df430 0x7f4c4c1386c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42076/0 (socket says 172.21.15.12:42076) 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.365+0000 7f4c4ad9d640 1 -- 172.21.15.12:0/2881326343 learned_addr learned my addr 172.21.15.12:0/2881326343 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c49d9b640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c140720 0x7f4c4c1deef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c4a59c640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c34004810 0x7f4c4c0bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c4ad9d640 1 -- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c34004810 msgr2=0x7f4c4c0bac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c4ad9d640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c34004810 0x7f4c4c0bac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c4ad9d640 1 -- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c140720 msgr2=0x7f4c4c1deef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c4ad9d640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c140720 0x7f4c4c1deef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c4ad9d640 1 -- 172.21.15.12:0/2881326343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c141e80 con 0x7f4c4c1df430 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c4ad9d640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c1df430 0x7f4c4c1386c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0bc470 tx=0x7f4c4400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c3affd640 1 -- 172.21.15.12:0/2881326343 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400eb60 con 0x7f4c4c1df430 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c3affd640 1 -- 172.21.15.12:0/2881326343 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4400ed00 con 0x7f4c4c1df430 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c50c45640 1 -- 172.21.15.12:0/2881326343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c13bd90 con 0x7f4c4c1df430 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c3affd640 1 -- 172.21.15.12:0/2881326343 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44014670 con 0x7f4c4c1df430 2024-04-27T07:04:25.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c50c45640 1 -- 172.21.15.12:0/2881326343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c140190 con 0x7f4c4c1df430 2024-04-27T07:04:25.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c50c45640 1 -- 172.21.15.12:0/2881326343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4c069330 con 0x7f4c4c1df430 2024-04-27T07:04:25.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c3affd640 1 -- 172.21.15.12:0/2881326343 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4c44014810 con 0x7f4c4c1df430 2024-04-27T07:04:25.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c3affd640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4c2c0683b0 0x7f4c2c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c3affd640 1 -- 172.21.15.12:0/2881326343 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4c440866d0 con 0x7f4c4c1df430 2024-04-27T07:04:25.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c4a59c640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4c2c0683b0 0x7f4c2c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.369+0000 7f4c4a59c640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4c2c0683b0 0x7f4c2c06a870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c13c830 tx=0x7f4c3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.373+0000 7f4c3affd640 1 -- 172.21.15.12:0/2881326343 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4c44056c80 con 0x7f4c4c1df430 2024-04-27T07:04:25.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.545+0000 7f4c50c45640 1 -- 172.21.15.12:0/2881326343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c4c144aa0 con 0x7f4c4c1df430 2024-04-27T07:04:25.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.549+0000 7f4c3affd640 1 -- 172.21.15.12:0/2881326343 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f4c44056580 con 0x7f4c4c1df430 2024-04-27T07:04:25.554 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:25.555 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 -- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4c2c0683b0 msgr2=0x7f4c2c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4c2c0683b0 0x7f4c2c06a870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c13c830 tx=0x7f4c3c009040 comp rx=0 tx=0).stop 2024-04-27T07:04:25.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 -- 172.21.15.12:0/2881326343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c1df430 msgr2=0x7f4c4c1386c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c1df430 0x7f4c4c1386c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0bc470 tx=0x7f4c4400e530 comp rx=0 tx=0).stop 2024-04-27T07:04:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 -- 172.21.15.12:0/2881326343 shutdown_connections 2024-04-27T07:04:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4c2c0683b0 0x7f4c2c06a870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c4c1df430 0x7f4c4c1386c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4c4c140720 0x7f4c4c1deef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 --2- 172.21.15.12:0/2881326343 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4c34004810 0x7f4c4c0bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 -- 172.21.15.12:0/2881326343 >> 172.21.15.12:0/2881326343 conn(0x7f4c4c0bb560 msgr2=0x7f4c4c143cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 -- 172.21.15.12:0/2881326343 shutdown_connections 2024-04-27T07:04:25.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.553+0000 7f4c50c45640 1 -- 172.21.15.12:0/2881326343 wait complete. 2024-04-27T07:04:25.575 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-27T07:04:25.575 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-27T07:04:25.575 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-27T07:04:25.576 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-27T07:04:25.576 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-27T07:04:25.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-27T07:04:25.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-27T07:04:25.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-27T07:04:25.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f4480e5a640 1 Processor -- start 2024-04-27T07:04:25.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f4480e5a640 1 -- start start 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f4480e5a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f447c09b5f0 0x7f447c141a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f4480e5a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f447c0af710 0x7f447c141f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f4480e5a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447c0ae890 con 0x7f447c09acd0 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f4480e5a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f447c0b03b0 con 0x7f447c0af710 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f4480e5a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f447c06e470 con 0x7f447c09b5f0 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f447ad9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f447c0af710 0x7f447c141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f447a59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f447c09acd0 0x7f447c0af000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40832/0 (socket says 172.21.15.12:40832) 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f447a59c640 1 -- 172.21.15.12:0/83645201 learned_addr learned my addr 172.21.15.12:0/83645201 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f4479d9b640 1 --2- 172.21.15.12:0/83645201 >> v2:172.21.15.53:3300/0 conn(0x7f447c09b5f0 0x7f447c141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f447ad9d640 1 -- 172.21.15.12:0/83645201 >> v2:172.21.15.53:3300/0 conn(0x7f447c09b5f0 msgr2=0x7f447c141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f447ad9d640 1 --2- 172.21.15.12:0/83645201 >> v2:172.21.15.53:3300/0 conn(0x7f447c09b5f0 0x7f447c141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f447ad9d640 1 -- 172.21.15.12:0/83645201 >> v1:172.21.15.12:6789/0 conn(0x7f447c09acd0 legacy=0x7f447c0af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.749+0000 7f447ad9d640 1 -- 172.21.15.12:0/83645201 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c1424c0 con 0x7f447c0af710 2024-04-27T07:04:25.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f447ad9d640 1 --2- 172.21.15.12:0/83645201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0af710 0x7f447c141f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f447c142750 tx=0x7f447001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc8b97166ee2558 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f447959a640 1 -- 172.21.15.12:0/83645201 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700b1070 con 0x7f447c0af710 2024-04-27T07:04:25.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f447959a640 1 -- 172.21.15.12:0/83645201 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44700ac440 con 0x7f447c0af710 2024-04-27T07:04:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f447959a640 1 -- 172.21.15.12:0/83645201 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44700ac810 con 0x7f447c0af710 2024-04-27T07:04:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f4480e5a640 1 -- 172.21.15.12:0/83645201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0af710 msgr2=0x7f447c141f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f4480e5a640 1 --2- 172.21.15.12:0/83645201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0af710 0x7f447c141f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f447c142750 tx=0x7f447001c260 comp rx=0 tx=0).stop 2024-04-27T07:04:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7fd68808d640 1 Processor -- start 2024-04-27T07:04:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f4480e5a640 1 -- 172.21.15.12:0/83645201 shutdown_connections 2024-04-27T07:04:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f4480e5a640 1 --2- 172.21.15.12:0/83645201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0af710 0x7f447c141f80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f4480e5a640 1 --2- 172.21.15.12:0/83645201 >> v2:172.21.15.53:3300/0 conn(0x7f447c09b5f0 0x7f447c141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f4480e5a640 1 -- 172.21.15.12:0/83645201 >> 172.21.15.12:0/83645201 conn(0x7f447c0b5e40 msgr2=0x7f447c0b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7fd68808d640 1 -- start start 2024-04-27T07:04:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68808d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd6800b6840 0x7fd6800b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68662a640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd68014c8f0 0x7fd680156ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40838/0 (socket says 172.21.15.12:40838) 2024-04-27T07:04:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68662a640 1 -- 172.21.15.12:0/826985285 learned_addr learned my addr 172.21.15.12:0/826985285 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd685e29640 1 --2- 172.21.15.12:0/826985285 >> v2:172.21.15.53:3300/0 conn(0x7fd6800b6840 0x7fd6800b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f4480e5a640 1 -- 172.21.15.12:0/83645201 shutdown_connections 2024-04-27T07:04:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.753+0000 7f4480e5a640 1 -- 172.21.15.12:0/83645201 wait complete. 2024-04-27T07:04:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f4480e5a640 1 Processor -- start 2024-04-27T07:04:25.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa1419ce640 1 Processor -- start 2024-04-27T07:04:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f32387b3640 1 Processor -- start 2024-04-27T07:04:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68808d640 1 -- 172.21.15.12:0/826985285 --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd68006f3e0 con 0x7fd68014c8f0 2024-04-27T07:04:25.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78eabe9640 1 Processor -- start 2024-04-27T07:04:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809613736 0 0) 0x7fd68006f3e0 con 0x7fd68014c8f0 2024-04-27T07:04:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68808d640 1 -- 172.21.15.12:0/826985285 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6800b6c20 con 0x7fd68014c240 2024-04-27T07:04:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68808d640 1 -- 172.21.15.12:0/826985285 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd68006f930 con 0x7fd6800b6840 2024-04-27T07:04:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd66c003530 con 0x7fd68014c8f0 2024-04-27T07:04:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2241567977 0 0) 0x7fd6800b6c20 con 0x7fd68014c240 2024-04-27T07:04:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd68006f3e0 con 0x7fd68014c240 2024-04-27T07:04:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2231217378 0 0) 0x7fd66c003530 con 0x7fd68014c8f0 2024-04-27T07:04:25.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 >> v2:172.21.15.53:3300/0 conn(0x7fd6800b6840 msgr2=0x7fd6800b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f32387b3640 1 -- start start 2024-04-27T07:04:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa1419ce640 1 -- start start 2024-04-27T07:04:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f32387b3640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3230073bb0 0x7f32300b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f32387b3640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f323013e640 0x7f323013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f32387b3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3230073f90 con 0x7f3230144bf0 2024-04-27T07:04:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f32387b3640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f323009dc90 con 0x7f3230073bb0 2024-04-27T07:04:25.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f32387b3640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f323009df70 con 0x7f323013e640 2024-04-27T07:04:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa1419ce640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa134006f00 0x7fa134036520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa1419ce640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa1340370b0 0x7fa1340c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa1419ce640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa134029d00 con 0x7fa134036c50 2024-04-27T07:04:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa1419ce640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa134036a60 con 0x7fa1340370b0 2024-04-27T07:04:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa1419ce640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fa134033f40 con 0x7fa134006f00 2024-04-27T07:04:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78eabe9640 1 -- start start 2024-04-27T07:04:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78eabe9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f78e40b6270 0x7f78e4148600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78eabe9640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f78e414b700 0x7f78e414bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78eabe9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f78e4141870 0x7f78e4141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78eabe9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f78e4148170 con 0x7f78e40b6270 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78eabe9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f78e40b6650 con 0x7f78e4141870 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78eabe9640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f78e4147e00 con 0x7f78e414b700 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 --2- 172.21.15.12:0/826985285 >> v2:172.21.15.53:3300/0 conn(0x7fd6800b6840 0x7fd6800b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 >> v1:172.21.15.53:6789/0 conn(0x7fd68014c240 legacy=0x7fd6800af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68006bcb0 con 0x7fd68014c8f0 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fd678001a20 con 0x7fd68014c8f0 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 >> v1:172.21.15.12:6789/0 conn(0x7fd68014c8f0 legacy=0x7fd680156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd66c003b50 0x7fd66c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd66c004810 0x7fd66c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd66c0052f0 0x7fd66c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd68006f930 con 0x7fd66c0052f0 2024-04-27T07:04:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f323654f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3230073bb0 0x7f32300b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f3235d4e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3230144bf0 0x7f3230145030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40854/0 (socket says 172.21.15.12:40854) 2024-04-27T07:04:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f3235d4e640 1 -- 172.21.15.12:0/549505364 learned_addr learned my addr 172.21.15.12:0/549505364 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f3236d50640 1 --2- 172.21.15.12:0/549505364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f323013e640 0x7f323013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f4480e5a640 1 -- start start 2024-04-27T07:04:25.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f4480e5a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f447c09acd0 0x7f447c0711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f4480e5a640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f447c09b5f0 0x7f447c1e7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f4480e5a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0af710 0x7f447c072a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f4480e5a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f447c155260 con 0x7f447c0af710 2024-04-27T07:04:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f4480e5a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1550e0 con 0x7f447c09acd0 2024-04-27T07:04:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f4480e5a640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f447c1553e0 con 0x7f447c09b5f0 2024-04-27T07:04:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f447ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0af710 0x7f447c072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f447ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0af710 0x7f447c072a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42106/0 (socket says 172.21.15.12:42106) 2024-04-27T07:04:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f447ad9d640 1 -- 172.21.15.12:0/1064336048 learned_addr learned my addr 172.21.15.12:0/1064336048 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa1409cc640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa134006f00 0x7fa134036520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c00c1a0 con 0x7fd66c004810 2024-04-27T07:04:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fd66c00c3a0 con 0x7fd66c003b50 2024-04-27T07:04:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd685e29640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd66c003b50 0x7fd66c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68662a640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd66c0052f0 0x7fd66c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa1409cc640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa134006f00 0x7fa134036520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:58512/0 (socket says 172.21.15.12:58512) 2024-04-27T07:04:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa13bfff640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fa134036c50 0x7fa134033830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:54818/0 (socket says 172.21.15.12:54818) 2024-04-27T07:04:25.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa13bfff640 1 -- 172.21.15.12:0/240257840 learned_addr learned my addr 172.21.15.12:0/240257840 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fa1411cd640 1 --2- 172.21.15.12:0/240257840 >> v2:172.21.15.53:3300/0 conn(0x7fa1340370b0 0x7fa1340c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78ea3e8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f78e4141870 0x7f78e4141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78ea3e8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f78e4141870 0x7f78e4141cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:37140/0 (socket says 172.21.15.12:37140) 2024-04-27T07:04:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78ea3e8640 1 -- 172.21.15.12:0/780575008 learned_addr learned my addr 172.21.15.12:0/780575008 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f4479d9b640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f447c09b5f0 0x7f447c1e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f447a59c640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f447c09acd0 0x7f447c0711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78e93e6640 1 --2- 172.21.15.12:0/780575008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f78e414b700 0x7f78e414bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f323654f640 1 -- 172.21.15.12:0/549505364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f323013e640 msgr2=0x7f323013ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f323654f640 1 --2- 172.21.15.12:0/549505364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f323013e640 0x7f323013ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f323654f640 1 -- 172.21.15.12:0/549505364 >> v1:172.21.15.12:6789/0 conn(0x7f3230144bf0 legacy=0x7f3230145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f323654f640 1 -- 172.21.15.12:0/549505364 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323009e250 con 0x7f3230073bb0 2024-04-27T07:04:25.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f4a1359e640 1 Processor -- start 2024-04-27T07:04:25.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f447a59c640 1 -- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f447c09b5f0 msgr2=0x7f447c1e7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f447a59c640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f447c09b5f0 0x7f447c1e7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f447a59c640 1 -- 172.21.15.12:0/1064336048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0af710 msgr2=0x7f447c072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f447a59c640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0af710 0x7f447c072a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f447a59c640 1 -- 172.21.15.12:0/1064336048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c0b3450 con 0x7f447c09acd0 2024-04-27T07:04:25.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f447ad9d640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c0af710 0x7f447c072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:04:25.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7f78e9be7640 1 --2- 172.21.15.12:0/780575008 >> v2:172.21.15.12:3300/0 conn(0x7f78e40b6270 0x7f78e4148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f78e9be7640 1 -- 172.21.15.12:0/780575008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f78e414b700 msgr2=0x7f78e414bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f78e9be7640 1 --2- 172.21.15.12:0/780575008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f78e414b700 0x7f78e414bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f78e9be7640 1 -- 172.21.15.12:0/780575008 >> v2:172.21.15.53:3300/0 conn(0x7f78e4141870 msgr2=0x7f78e4141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f78e9be7640 1 --2- 172.21.15.12:0/780575008 >> v2:172.21.15.53:3300/0 conn(0x7f78e4141870 0x7f78e4141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fa1411cd640 1 -- 172.21.15.12:0/240257840 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa134006f00 msgr2=0x7fa134036520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f78e9be7640 1 -- 172.21.15.12:0/780575008 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e40b9cd0 con 0x7f78e40b6270 2024-04-27T07:04:25.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fa1411cd640 1 --2- 172.21.15.12:0/240257840 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa134006f00 0x7fa134036520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fa1411cd640 1 -- 172.21.15.12:0/240257840 >> v1:172.21.15.53:6789/0 conn(0x7fa134036c50 legacy=0x7fa134033830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fa1411cd640 1 -- 172.21.15.12:0/240257840 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1340362b0 con 0x7fa1340370b0 2024-04-27T07:04:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f447a59c640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f447c09acd0 0x7f447c0711b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f447400cb20 tx=0x7f44740020d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f446affd640 1 -- 172.21.15.12:0/1064336048 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474018070 con 0x7f447c09acd0 2024-04-27T07:04:25.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1359e640 1 -- start start 2024-04-27T07:04:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1359e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a140b2620 0x7f4a14143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1359e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a14073420 0x7f4a1414e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1359e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a140b2a00 con 0x7f4a141440f0 2024-04-27T07:04:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1359e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f4a14074150 con 0x7f4a140b2620 2024-04-27T07:04:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1359e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f4a140705a0 con 0x7f4a14073420 2024-04-27T07:04:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a140b2620 0x7f4a14143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a140b2620 0x7f4a14143bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42122/0 (socket says 172.21.15.12:42122) 2024-04-27T07:04:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1259c640 1 -- 172.21.15.12:0/3778616381 learned_addr learned my addr 172.21.15.12:0/3778616381 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f446affd640 1 -- 172.21.15.12:0/1064336048 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f447400a700 con 0x7f447c09acd0 2024-04-27T07:04:25.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f446affd640 1 -- 172.21.15.12:0/1064336048 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447400aad0 con 0x7f447c09acd0 2024-04-27T07:04:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68662a640 1 -- 172.21.15.12:0/826985285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd66c003b50 msgr2=0x7fd66c003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68662a640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd66c003b50 0x7fd66c003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68662a640 1 -- 172.21.15.12:0/826985285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd66c004810 msgr2=0x7fd66c004c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:04:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68662a640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd66c004810 0x7fd66c004c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd68662a640 1 -- 172.21.15.12:0/826985285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68006bcb0 con 0x7fd66c0052f0 2024-04-27T07:04:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.757+0000 7fd685e29640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd66c003b50 0x7fd66c003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:04:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fd68662a640 1 -- 172.21.15.12:0/826985285 reap_dead start 2024-04-27T07:04:25.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fd68662a640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd66c0052f0 0x7fd66c00b9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd68009d360 tx=0x7fd67801d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5027e20f9471595 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f323654f640 1 --2- 172.21.15.12:0/549505364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3230073bb0 0x7f32300b5740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f322c0097c0 tx=0x7f322c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=907d61a1ed367e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f323554d640 1 -- 172.21.15.12:0/549505364 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c0b1070 con 0x7f3230073bb0 2024-04-27T07:04:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f323554d640 1 -- 172.21.15.12:0/549505364 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322c0ac410 con 0x7f3230073bb0 2024-04-27T07:04:25.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4480e5a640 1 -- 172.21.15.12:0/1064336048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c073560 con 0x7f447c09acd0 2024-04-27T07:04:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4480e5a640 1 -- 172.21.15.12:0/1064336048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c147eb0 con 0x7f447c09acd0 2024-04-27T07:04:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1259c640 1 -- 172.21.15.12:0/3778616381 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a14073420 msgr2=0x7f4a1414e450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:04:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1259c640 1 --2- 172.21.15.12:0/3778616381 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a14073420 0x7f4a1414e450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1259c640 1 -- 172.21.15.12:0/3778616381 >> v1:172.21.15.53:6789/0 conn(0x7f4a141440f0 legacy=0x7f4a14072d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:04:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f4a1259c640 1 -- 172.21.15.12:0/3778616381 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a140b0ec0 con 0x7f4a140b2620 2024-04-27T07:04:25.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f32387b3640 1 -- 172.21.15.12:0/549505364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3230073bb0 msgr2=0x7f32300b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f32387b3640 1 --2- 172.21.15.12:0/549505364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3230073bb0 0x7f32300b5740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f322c0097c0 tx=0x7f322c01c4a0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fa1411cd640 1 --2- 172.21.15.12:0/240257840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1340370b0 0x7fa1340c8590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa1280097c0 tx=0x7fa12801c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30fa7555d271a266 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f32387b3640 1 -- 172.21.15.12:0/549505364 shutdown_connections 2024-04-27T07:04:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f32387b3640 1 --2- 172.21.15.12:0/549505364 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f323013e640 0x7f323013ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f32387b3640 1 --2- 172.21.15.12:0/549505364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3230073bb0 0x7f32300b5740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f32387b3640 1 -- 172.21.15.12:0/549505364 >> 172.21.15.12:0/549505364 conn(0x7f32300b9340 msgr2=0x7f32300b9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f32387b3640 1 -- 172.21.15.12:0/549505364 shutdown_connections 2024-04-27T07:04:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67001f040 con 0x7fd66c0052f0 2024-04-27T07:04:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6780072d0 con 0x7fd66c0052f0 2024-04-27T07:04:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fd684e27640 1 -- 172.21.15.12:0/826985285 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67800a500 con 0x7fd66c0052f0 2024-04-27T07:04:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fd68808d640 1 -- 172.21.15.12:0/826985285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd66c0052f0 msgr2=0x7fd66c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7fd68808d640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd66c0052f0 0x7fd66c00b9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd68009d360 tx=0x7fd67801d320 comp rx=0 tx=0).stop 2024-04-27T07:04:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 -- 172.21.15.12:0/826985285 shutdown_connections 2024-04-27T07:04:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 --2- 172.21.15.12:0/826985285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd66c0052f0 0x7fd66c00b9d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 -- 172.21.15.12:0/826985285 >> 172.21.15.12:0/826985285 conn(0x7fd6800b6ff0 msgr2=0x7fd6800b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4480e5a640 1 -- 172.21.15.12:0/1064336048 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f447c1e7ee0 con 0x7f447c09acd0 2024-04-27T07:04:25.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 -- 172.21.15.12:0/826985285 shutdown_connections 2024-04-27T07:04:25.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f78e9be7640 1 --2- 172.21.15.12:0/780575008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e40b6270 0x7f78e4148600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f78e40b4c90 tx=0x7f78d801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2be2139b5f9b16d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa13b7fe640 1 -- 172.21.15.12:0/240257840 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1280b1070 con 0x7fa1340370b0 2024-04-27T07:04:25.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa13b7fe640 1 -- 172.21.15.12:0/240257840 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1280ac440 con 0x7fa1340370b0 2024-04-27T07:04:25.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1259c640 1 --2- 172.21.15.12:0/3778616381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a140b2620 0x7f4a14143bb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400d370 tx=0x7f4a0401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d862a3190745a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.761+0000 7f32387b3640 1 -- 172.21.15.12:0/549505364 wait complete. 2024-04-27T07:04:25.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f32387b3640 1 Processor -- start 2024-04-27T07:04:25.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa13b7fe640 1 -- 172.21.15.12:0/240257840 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1280ac810 con 0x7fa1340370b0 2024-04-27T07:04:25.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa1419ce640 1 -- 172.21.15.12:0/240257840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1340370b0 msgr2=0x7fa1340c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa1419ce640 1 --2- 172.21.15.12:0/240257840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1340370b0 0x7fa1340c8590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa1280097c0 tx=0x7fa12801c320 comp rx=0 tx=0).stop 2024-04-27T07:04:25.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa1419ce640 1 -- 172.21.15.12:0/240257840 shutdown_connections 2024-04-27T07:04:25.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa1419ce640 1 --2- 172.21.15.12:0/240257840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1340370b0 0x7fa1340c8590 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa1419ce640 1 --2- 172.21.15.12:0/240257840 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa134006f00 0x7fa134036520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa1419ce640 1 -- 172.21.15.12:0/240257840 >> 172.21.15.12:0/240257840 conn(0x7fa13403a890 msgr2=0x7fa13403aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 -- 172.21.15.12:0/826985285 wait complete. 2024-04-27T07:04:25.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 Processor -- start 2024-04-27T07:04:25.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa1419ce640 1 -- 172.21.15.12:0/240257840 shutdown_connections 2024-04-27T07:04:25.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1159a640 1 -- 172.21.15.12:0/3778616381 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a040ad070 con 0x7f4a140b2620 2024-04-27T07:04:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1159a640 1 -- 172.21.15.12:0/3778616381 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a040a7b20 con 0x7f4a140b2620 2024-04-27T07:04:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1159a640 1 -- 172.21.15.12:0/3778616381 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400be60 con 0x7f4a140b2620 2024-04-27T07:04:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f446affd640 1 -- 172.21.15.12:0/1064336048 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f447400ac70 con 0x7f447c09acd0 2024-04-27T07:04:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f446affd640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f445c0683b0 0x7f445c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f446affd640 1 -- 172.21.15.12:0/1064336048 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4474086610 con 0x7f447c09acd0 2024-04-27T07:04:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f446affd640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] conn(0x7f445c070630 0x7f445c072a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f446affd640 1 -- 172.21.15.12:0/1064336048 --> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f445c073100 con 0x7f445c070630 2024-04-27T07:04:25.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f446affd640 1 -- 172.21.15.12:0/1064336048 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4474086a00 con 0x7f447c09acd0 2024-04-27T07:04:25.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4479d9b640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f445c0683b0 0x7f445c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f32387b3640 1 -- start start 2024-04-27T07:04:25.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f32387b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3230073bb0 0x7f32301303b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f32387b3640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f323013e640 0x7f32300b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f32387b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230144bf0 0x7f3230142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f32387b3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f323009b690 con 0x7f3230144bf0 2024-04-27T07:04:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f32387b3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f323009b510 con 0x7f3230073bb0 2024-04-27T07:04:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f32387b3640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f323009b810 con 0x7f323013e640 2024-04-27T07:04:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f323654f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3230073bb0 0x7f32301303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f323654f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3230073bb0 0x7f32301303b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:37150/0 (socket says 172.21.15.12:37150) 2024-04-27T07:04:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f323654f640 1 -- 172.21.15.12:0/1697048584 learned_addr learned my addr 172.21.15.12:0/1697048584 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1359e640 1 -- 172.21.15.12:0/3778616381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a140b2620 msgr2=0x7f4a14143bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1359e640 1 --2- 172.21.15.12:0/3778616381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a140b2620 0x7f4a14143bb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400d370 tx=0x7f4a0401ec90 comp rx=0 tx=0).stop 2024-04-27T07:04:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa1419ce640 1 -- 172.21.15.12:0/240257840 wait complete. 2024-04-27T07:04:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 -- start start 2024-04-27T07:04:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd66c0052f0 0x7fd68014ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd68014f3a0 0x7fd680148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68014f820 0x7fd6801487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd68006c890 con 0x7fd68014f820 2024-04-27T07:04:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd68006c270 con 0x7fd66c0052f0 2024-04-27T07:04:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68808d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fd68006ca10 con 0x7fd68014f3a0 2024-04-27T07:04:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1359e640 1 -- 172.21.15.12:0/3778616381 shutdown_connections 2024-04-27T07:04:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f3235d4e640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f323013e640 0x7f32300b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f3236d50640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230144bf0 0x7f3230142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1359e640 1 --2- 172.21.15.12:0/3778616381 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a14073420 0x7f4a1414e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1359e640 1 --2- 172.21.15.12:0/3778616381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a140b2620 0x7f4a14143bb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1359e640 1 -- 172.21.15.12:0/3778616381 >> 172.21.15.12:0/3778616381 conn(0x7f4a140b9310 msgr2=0x7f4a140b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f447ad9d640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] conn(0x7f445c070630 0x7f445c072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd685e29640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd66c0052f0 0x7fd68014ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd685e29640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd66c0052f0 0x7fd68014ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:37156/0 (socket says 172.21.15.12:37156) 2024-04-27T07:04:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd685e29640 1 -- 172.21.15.12:0/310315404 learned_addr learned my addr 172.21.15.12:0/310315404 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fd68662a640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68014f820 0x7fd6801487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa1419ce640 1 Processor -- start 2024-04-27T07:04:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f78e8be5640 1 -- 172.21.15.12:0/780575008 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d80b0070 con 0x7f78e40b6270 2024-04-27T07:04:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7fa1419ce640 1 -- start start 2024-04-27T07:04:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f78e8be5640 1 -- 172.21.15.12:0/780575008 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d80ab600 con 0x7f78e40b6270 2024-04-27T07:04:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f78e8be5640 1 -- 172.21.15.12:0/780575008 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d80ab9d0 con 0x7f78e40b6270 2024-04-27T07:04:25.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4a1359e640 1 -- 172.21.15.12:0/3778616381 shutdown_connections 2024-04-27T07:04:25.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f3236d50640 1 -- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f323013e640 msgr2=0x7f32300b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f3236d50640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f323013e640 0x7f32300b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f3236d50640 1 -- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3230073bb0 msgr2=0x7f32301303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f4479d9b640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f445c0683b0 0x7f445c06a870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f447c1453e0 tx=0x7f446c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.765+0000 7f3236d50640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3230073bb0 0x7f32301303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f3236d50640 1 -- 172.21.15.12:0/1697048584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32300ae360 con 0x7f3230144bf0 2024-04-27T07:04:25.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a1359e640 1 -- 172.21.15.12:0/3778616381 wait complete. 2024-04-27T07:04:25.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd685628640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd68014f3a0 0x7fd680148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd68662a640 1 -- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd68014f3a0 msgr2=0x7fd680148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd68662a640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd68014f3a0 0x7fd680148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd68662a640 1 -- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd66c0052f0 msgr2=0x7fd68014ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd68662a640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd66c0052f0 0x7fd68014ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd68662a640 1 -- 172.21.15.12:0/310315404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68014d3a0 con 0x7fd68014f820 2024-04-27T07:04:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd685628640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd68014f3a0 0x7fd680148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 -- 172.21.15.12:0/780575008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e40b6270 msgr2=0x7f78e4148600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 --2- 172.21.15.12:0/780575008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e40b6270 0x7f78e4148600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f78e40b4c90 tx=0x7f78d801c3f0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd68662a640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68014f820 0x7fd6801487d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd6800ba4c0 tx=0x7fd66c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1419ce640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa134006f00 0x7fa134175480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1419ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa134036c50 0x7fa13403ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1419ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1340370b0 0x7fa1340bef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1419ce640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa1340cf660 con 0x7fa1340370b0 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1419ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa1340d3f80 con 0x7fa134036c50 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1419ce640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fa1340cf7e0 con 0x7fa134006f00 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1411cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1340370b0 0x7fa1340bef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1411cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1340370b0 0x7fa1340bef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42142/0 (socket says 172.21.15.12:42142) 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1411cd640 1 -- 172.21.15.12:0/1675495849 learned_addr learned my addr 172.21.15.12:0/1675495849 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1411cd640 1 -- 172.21.15.12:0/1675495849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa134006f00 msgr2=0x7fa134175480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1411cd640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa134006f00 0x7fa134175480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1411cd640 1 -- 172.21.15.12:0/1675495849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa134036c50 msgr2=0x7fa13403ccf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1411cd640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa134036c50 0x7fa13403ccf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1411cd640 1 -- 172.21.15.12:0/1675495849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1340c18c0 con 0x7fa1340370b0 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1411cd640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1340370b0 0x7fa1340bef10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1280af040 tx=0x7fa1280190c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 -- 172.21.15.12:0/780575008 shutdown_connections 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 --2- 172.21.15.12:0/780575008 >> v2:172.21.15.53:3300/0 conn(0x7f78e4141870 0x7f78e4141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 --2- 172.21.15.12:0/780575008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f78e414b700 0x7f78e414bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 --2- 172.21.15.12:0/780575008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e40b6270 0x7f78e4148600 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 -- 172.21.15.12:0/780575008 >> 172.21.15.12:0/780575008 conn(0x7f78e4068190 msgr2=0x7f78e40bd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 -- 172.21.15.12:0/780575008 shutdown_connections 2024-04-27T07:04:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 -- 172.21.15.12:0/780575008 wait complete. 2024-04-27T07:04:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd6767fc640 1 -- 172.21.15.12:0/310315404 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6780070d0 con 0x7fd68014f820 2024-04-27T07:04:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd6767fc640 1 -- 172.21.15.12:0/310315404 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd678004b70 con 0x7fd68014f820 2024-04-27T07:04:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd6767fc640 1 -- 172.21.15.12:0/310315404 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678007480 con 0x7fd68014f820 2024-04-27T07:04:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f3236d50640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230144bf0 0x7f3230142880 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3228007d90 tx=0x7f322800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f3226ffd640 1 -- 172.21.15.12:0/1697048584 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322800edd0 con 0x7f3230144bf0 2024-04-27T07:04:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f3226ffd640 1 -- 172.21.15.12:0/1697048584 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32280039a0 con 0x7f3230144bf0 2024-04-27T07:04:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f3226ffd640 1 -- 172.21.15.12:0/1697048584 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228003d90 con 0x7f3230144bf0 2024-04-27T07:04:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3230130fc0 con 0x7f3230144bf0 2024-04-27T07:04:25.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32300ae8e0 con 0x7f3230144bf0 2024-04-27T07:04:25.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3230068c20 con 0x7f3230144bf0 2024-04-27T07:04:25.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a1359e640 1 Processor -- start 2024-04-27T07:04:25.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f447ad9d640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] conn(0x7f445c070630 0x7f445c072a50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1397fa640 1 -- 172.21.15.12:0/1675495849 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1280b1040 con 0x7fa1340370b0 2024-04-27T07:04:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1397fa640 1 -- 172.21.15.12:0/1675495849 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1280ba070 con 0x7fa1340370b0 2024-04-27T07:04:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1397fa640 1 -- 172.21.15.12:0/1675495849 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1280b4850 con 0x7fa1340370b0 2024-04-27T07:04:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 Processor -- start 2024-04-27T07:04:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 -- start start 2024-04-27T07:04:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68009e480 con 0x7fd68014f820 2024-04-27T07:04:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6800b3460 con 0x7fd68014f820 2024-04-27T07:04:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd68009df50 con 0x7fd68014f820 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f804da5b640 1 Processor -- start 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f804da5b640 1 -- start start 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f804da5b640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80400361f0 0x7f80400d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f804da5b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8040036d80 con 0x7f80400369a0 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f804da5b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80400352e0 con 0x7f80400d0e00 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f804da5b640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f80400d17c0 con 0x7f80400361f0 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f804ca59640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f80400369a0 0x7f8040039860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40868/0 (socket says 172.21.15.12:40868) 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f804ca59640 1 -- 172.21.15.12:0/3552490238 learned_addr learned my addr 172.21.15.12:0/3552490238 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f804d25a640 1 --2- 172.21.15.12:0/3552490238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80400361f0 0x7f80400d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f80477fe640 1 -- 172.21.15.12:0/3552490238 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4030074707 0 0) 0x7f8040036d80 con 0x7f80400369a0 2024-04-27T07:04:25.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f80477fe640 1 -- 172.21.15.12:0/3552490238 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8038003530 con 0x7f80400369a0 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a1359e640 1 -- start start 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a1359e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a14073420 0x7f4a1413d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a1359e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a140b2620 0x7f4a14132330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a1359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a141440f0 0x7f4a14138ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a1359e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14151540 con 0x7f4a141440f0 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a1359e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141513c0 con 0x7f4a14073420 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a1359e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f4a141516c0 con 0x7f4a140b2620 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a11d9b640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a140b2620 0x7f4a14132330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a12d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a141440f0 0x7f4a14138ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a12d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a141440f0 0x7f4a14138ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42154/0 (socket says 172.21.15.12:42154) 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a12d9d640 1 -- 172.21.15.12:0/3900620592 learned_addr learned my addr 172.21.15.12:0/3900620592 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a1259c640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a14073420 0x7f4a1413d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f3226ffd640 1 -- 172.21.15.12:0/1697048584 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f322801b020 con 0x7f3230144bf0 2024-04-27T07:04:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f804d25a640 1 -- 172.21.15.12:0/3552490238 >> v1:172.21.15.53:6789/0 conn(0x7f80400d0e00 legacy=0x7f8040035ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f804d25a640 1 -- 172.21.15.12:0/3552490238 >> v1:172.21.15.12:6789/0 conn(0x7f80400369a0 legacy=0x7f8040039860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f804d25a640 1 -- 172.21.15.12:0/3552490238 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80400355c0 con 0x7f80400361f0 2024-04-27T07:04:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f4a12d9d640 1 -- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a140b2620 msgr2=0x7f4a14132330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a12d9d640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4a140b2620 0x7f4a14132330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a12d9d640 1 -- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a14073420 msgr2=0x7f4a1413d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a12d9d640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a14073420 0x7f4a1413d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a12d9d640 1 -- 172.21.15.12:0/3900620592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a14132c70 con 0x7f4a141440f0 2024-04-27T07:04:25.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f446affd640 1 -- 172.21.15.12:0/1064336048 <== osd.7 v2:172.21.15.53:6816/2338388162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f445c073100 con 0x7f445c070630 2024-04-27T07:04:25.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7f78eabe9640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f78e40b6270 0x7f78e4151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78eabe9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e4141870 0x7f78e41e0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78eabe9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e414b700 0x7f78e414ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78eabe9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f78e40b5ad0 con 0x7f78e4141870 2024-04-27T07:04:25.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78eabe9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f78e40b5950 con 0x7f78e414b700 2024-04-27T07:04:25.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78eabe9640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f78e40b5c50 con 0x7f78e40b6270 2024-04-27T07:04:25.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f3226ffd640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3214068480 0x7f321406a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78ea3e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e414b700 0x7f78e414ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f3226ffd640 1 -- 172.21.15.12:0/1697048584 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f32280877e0 con 0x7f3230144bf0 2024-04-27T07:04:25.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a12d9d640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a141440f0 0x7f4a14138ca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00b410 tx=0x7f4a0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78ea3e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e414b700 0x7f78e414ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:37164/0 (socket says 172.21.15.12:37164) 2024-04-27T07:04:25.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78ea3e8640 1 -- 172.21.15.12:0/155423390 learned_addr learned my addr 172.21.15.12:0/155423390 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78e93e6640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e4141870 0x7f78e41e0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa134175aa0 con 0x7fa1340370b0 2024-04-27T07:04:25.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.769+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1340bb720 con 0x7fa1340370b0 2024-04-27T07:04:25.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f3226ffd640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] conn(0x7f3214070700 0x7f3214072b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f3226ffd640 1 -- 172.21.15.12:0/1697048584 --> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32140731d0 con 0x7f3214070700 2024-04-27T07:04:25.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f3226ffd640 1 -- 172.21.15.12:0/1697048584 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3228087bd0 con 0x7f3230144bf0 2024-04-27T07:04:25.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa134006ce0 con 0x7fa1340370b0 2024-04-27T07:04:25.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78e93e6640 1 -- 172.21.15.12:0/155423390 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f78e40b6270 msgr2=0x7f78e4151e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-27T07:04:25.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78e93e6640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f78e40b6270 0x7f78e4151e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78e93e6640 1 -- 172.21.15.12:0/155423390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e414b700 msgr2=0x7f78e414ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a02ffd640 1 -- 172.21.15.12:0/3900620592 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c015020 con 0x7f4a141440f0 2024-04-27T07:04:25.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a02ffd640 1 -- 172.21.15.12:0/3900620592 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a0c011830 con 0x7f4a141440f0 2024-04-27T07:04:25.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a02ffd640 1 -- 172.21.15.12:0/3900620592 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c011c60 con 0x7f4a141440f0 2024-04-27T07:04:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78e93e6640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e414b700 0x7f78e414ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78e93e6640 1 -- 172.21.15.12:0/155423390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e41513f0 con 0x7f78e4141870 2024-04-27T07:04:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78ea3e8640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78e414b700 0x7f78e414ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:04:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78e93e6640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e4141870 0x7f78e41e0bc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78e000d5f0 tx=0x7f78e000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f3235d4e640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] conn(0x7f3214070700 0x7f3214072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f3235d4e640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] conn(0x7f3214070700 0x7f3214072b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a14136620 con 0x7f4a141440f0 2024-04-27T07:04:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1413a040 con 0x7f4a141440f0 2024-04-27T07:04:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f323654f640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3214068480 0x7f321406a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f804d25a640 1 --2- 172.21.15.12:0/3552490238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80400361f0 0x7f80400d1280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8048050cf0 tx=0x7f8048097de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d5e21d5088c7f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a14068c20 con 0x7f4a141440f0 2024-04-27T07:04:25.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78d67fc640 1 -- 172.21.15.12:0/155423390 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e00125f0 con 0x7f78e4141870 2024-04-27T07:04:25.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78d67fc640 1 -- 172.21.15.12:0/155423390 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e0019070 con 0x7f78e4141870 2024-04-27T07:04:25.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f323654f640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3214068480 0x7f321406a940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f322c01a0d0 tx=0x7f322c0a6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f3226ffd640 1 -- 172.21.15.12:0/1697048584 <== osd.4 v2:172.21.15.53:6800/151807054 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32140731d0 con 0x7f3214070700 2024-04-27T07:04:25.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f80477fe640 1 -- 172.21.15.12:0/3552490238 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048120b80 con 0x7f80400361f0 2024-04-27T07:04:25.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78d67fc640 1 -- 172.21.15.12:0/155423390 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0012990 con 0x7f78e4141870 2024-04-27T07:04:25.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e414dc80 con 0x7f78e4141870 2024-04-27T07:04:25.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f4a02ffd640 1 -- 172.21.15.12:0/3900620592 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4a0c018020 con 0x7f4a141440f0 2024-04-27T07:04:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e41e03e0 con 0x7f78e4141870 2024-04-27T07:04:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78e4069330 con 0x7f78e4141870 2024-04-27T07:04:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7fa1397fa640 1 -- 172.21.15.12:0/1675495849 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa1280b49f0 con 0x7fa1340370b0 2024-04-27T07:04:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fa1397fa640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa120068450 0x7fa12006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fa1397fa640 1 -- 172.21.15.12:0/1675495849 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa1281213c0 con 0x7fa1340370b0 2024-04-27T07:04:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fa1397fa640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] conn(0x7fa1200706d0 0x7fa120072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fa1397fa640 1 -- 172.21.15.12:0/1675495849 --> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa128008eb0 con 0x7fa1200706d0 2024-04-27T07:04:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fa1397fa640 1 -- 172.21.15.12:0/1675495849 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa1281217b0 con 0x7fa1340370b0 2024-04-27T07:04:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f4a02ffd640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f49f40683d0 0x7f49f406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f4a02ffd640 1 -- 172.21.15.12:0/3900620592 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a0c086cd0 con 0x7f4a141440f0 2024-04-27T07:04:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f4a02ffd640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] conn(0x7f49f4070650 0x7f49f4072a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f4a02ffd640 1 -- 172.21.15.12:0/3900620592 --> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a0c00de50 con 0x7f49f4070650 2024-04-27T07:04:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f4a02ffd640 1 -- 172.21.15.12:0/3900620592 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4a0c0871b0 con 0x7f4a141440f0 2024-04-27T07:04:25.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f80477fe640 1 -- 172.21.15.12:0/3552490238 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8048120d60 con 0x7f80400361f0 2024-04-27T07:04:25.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.773+0000 7f80477fe640 1 -- 172.21.15.12:0/3552490238 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048128640 con 0x7f80400361f0 2024-04-27T07:04:25.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f804da5b640 1 -- 172.21.15.12:0/3552490238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80400361f0 msgr2=0x7f80400d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f804da5b640 1 --2- 172.21.15.12:0/3552490238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80400361f0 0x7f80400d1280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8048050cf0 tx=0x7f8048097de0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fa13bfff640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] conn(0x7fa1200706d0 0x7fa120072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f4a11d9b640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] conn(0x7f49f4070650 0x7f49f4072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f4a1259c640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f49f40683d0 0x7f49f406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fa13bfff640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] conn(0x7fa1200706d0 0x7fa120072af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f4a11d9b640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] conn(0x7f49f4070650 0x7f49f4072a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fd6767fc640 1 -- 172.21.15.12:0/310315404 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd6780b6030 con 0x7fd68014f820 2024-04-27T07:04:25.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fd6767fc640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd664068480 0x7fd66406a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fd6767fc640 1 -- 172.21.15.12:0/310315404 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd6781203b0 con 0x7fd68014f820 2024-04-27T07:04:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fd6767fc640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] conn(0x7fd6640706a0 0x7fd664072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fd6767fc640 1 -- 172.21.15.12:0/310315404 --> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd678005ab0 con 0x7fd6640706a0 2024-04-27T07:04:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fd6767fc640 1 -- 172.21.15.12:0/310315404 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd6781207a0 con 0x7fd68014f820 2024-04-27T07:04:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f4a1259c640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f49f40683d0 0x7f49f406a890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a040b2000 tx=0x7f4a0400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fd685e29640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd664068480 0x7fd66406a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f78d67fc640 1 -- 172.21.15.12:0/155423390 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f78e0012b30 con 0x7f78e4141870 2024-04-27T07:04:25.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fa1397fa640 1 -- 172.21.15.12:0/1675495849 <== osd.0 v2:172.21.15.12:6813/3217679893 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa128008eb0 con 0x7fa1200706d0 2024-04-27T07:04:25.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f78d67fc640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f78c8068380 0x7f78c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7fd685e29640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd664068480 0x7fd66406a940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6700045a0 tx=0x7fd670020040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f78d67fc640 1 -- 172.21.15.12:0/155423390 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f78e00877d0 con 0x7f78e4141870 2024-04-27T07:04:25.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f78d67fc640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] conn(0x7f78c8070520 0x7f78c8072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f78d67fc640 1 -- 172.21.15.12:0/155423390 --> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78e0001d40 con 0x7f78c8070520 2024-04-27T07:04:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f78d67fc640 1 -- 172.21.15.12:0/155423390 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f78e0087b80 con 0x7f78e4141870 2024-04-27T07:04:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f78e9be7640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f78c8068380 0x7f78c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.777+0000 7f78ea3e8640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] conn(0x7f78c8070520 0x7f78c8072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7fd685628640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] conn(0x7fd6640706a0 0x7fd664072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f78ea3e8640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] conn(0x7f78c8070520 0x7f78c8072940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 -- 172.21.15.12:0/3552490238 shutdown_connections 2024-04-27T07:04:25.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 --2- 172.21.15.12:0/3552490238 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80400361f0 0x7f80400d1280 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 -- 172.21.15.12:0/3552490238 >> 172.21.15.12:0/3552490238 conn(0x7f804003a890 msgr2=0x7f804003aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7fd685628640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] conn(0x7fd6640706a0 0x7fd664072ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 -- 172.21.15.12:0/3552490238 shutdown_connections 2024-04-27T07:04:25.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f78e9be7640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f78c8068380 0x7f78c806a840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78d80ad000 tx=0x7f78d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f78d67fc640 1 -- 172.21.15.12:0/155423390 <== osd.1 v2:172.21.15.12:6800/3260695366 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78e0001d40 con 0x7f78c8070520 2024-04-27T07:04:25.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f4a02ffd640 1 -- 172.21.15.12:0/3900620592 <== osd.6 v2:172.21.15.53:6808/289427539 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a0c00de50 con 0x7f49f4070650 2024-04-27T07:04:25.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7fd6767fc640 1 -- 172.21.15.12:0/310315404 <== osd.2 v2:172.21.15.12:6811/3457925485 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd678005ab0 con 0x7fd6640706a0 2024-04-27T07:04:25.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 -- 172.21.15.12:0/3552490238 wait complete. 2024-04-27T07:04:25.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 Processor -- start 2024-04-27T07:04:25.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 -- start start 2024-04-27T07:04:25.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400361f0 0x7f80400c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80400369a0 0x7f80400c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80400d0e00 0x7f80400cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f80400a62b0 con 0x7f80400361f0 2024-04-27T07:04:25.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f80400a6130 con 0x7f80400369a0 2024-04-27T07:04:25.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804da5b640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f80400a6430 con 0x7f80400d0e00 2024-04-27T07:04:25.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f8047fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80400369a0 0x7f80400c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f8047fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80400369a0 0x7f80400c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:37174/0 (socket says 172.21.15.12:37174) 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f8047fff640 1 -- 172.21.15.12:0/3890030062 learned_addr learned my addr 172.21.15.12:0/3890030062 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.781+0000 7f804d25a640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80400d0e00 0x7f80400cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f8047fff640 1 -- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80400d0e00 msgr2=0x7f80400cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f8047fff640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f80400d0e00 0x7f80400cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f8047fff640 1 -- 172.21.15.12:0/3890030062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400361f0 msgr2=0x7f80400c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f8047fff640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80400361f0 0x7f80400c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f8047fff640 1 -- 172.21.15.12:0/3890030062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80400a6b40 con 0x7f80400369a0 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f8047fff640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80400369a0 0x7f80400c6350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f803400b410 tx=0x7f803400b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f80457fa640 1 -- 172.21.15.12:0/3890030062 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034015030 con 0x7f80400369a0 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f80457fa640 1 -- 172.21.15.12:0/3890030062 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8034004020 con 0x7f80400369a0 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f80457fa640 1 -- 172.21.15.12:0/3890030062 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034004410 con 0x7f80400369a0 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f804da5b640 1 -- 172.21.15.12:0/3890030062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80400c69c0 con 0x7f80400369a0 2024-04-27T07:04:25.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f804da5b640 1 -- 172.21.15.12:0/3890030062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80400ca2c0 con 0x7f80400369a0 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.785+0000 7f804da5b640 1 -- 172.21.15.12:0/3890030062 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80400dc020 con 0x7f80400369a0 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.789+0000 7f80457fa640 1 -- 172.21.15.12:0/3890030062 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8034018070 con 0x7f80400369a0 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.789+0000 7f80457fa640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f802c0683b0 0x7f802c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.789+0000 7f80457fa640 1 -- 172.21.15.12:0/3890030062 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8034086e50 con 0x7f80400369a0 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.789+0000 7f80457fa640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] conn(0x7f802c0705b0 0x7f802c0729d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.789+0000 7f804ca59640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f802c0683b0 0x7f802c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.789+0000 7f80457fa640 1 -- 172.21.15.12:0/3890030062 --> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f803401be50 con 0x7f802c0705b0 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.789+0000 7f80457fa640 1 -- 172.21.15.12:0/3890030062 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8034087200 con 0x7f80400369a0 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.789+0000 7f804d25a640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] conn(0x7f802c0705b0 0x7f802c0729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.789+0000 7f804ca59640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f802c0683b0 0x7f802c06a870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f803c007fd0 tx=0x7f803c007940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.789+0000 7f804d25a640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] conn(0x7f802c0705b0 0x7f802c0729d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.793+0000 7fa1409cc640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa120068450 0x7fa12006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.793+0000 7f80457fa640 1 -- 172.21.15.12:0/3890030062 <== osd.5 v2:172.21.15.53:6824/2061790692 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f803401be50 con 0x7f802c0705b0 2024-04-27T07:04:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.793+0000 7fa1409cc640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa120068450 0x7fa12006a910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1300099f0 tx=0x7fa130009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 --> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3230145160 con 0x7f3214070700 2024-04-27T07:04:25.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f3226ffd640 1 -- 172.21.15.12:0/1697048584 <== osd.4 v2:172.21.15.53:6800/151807054 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3230145160 con 0x7f3214070700 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] conn(0x7f3214070700 msgr2=0x7f3214072b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] conn(0x7f3214070700 0x7f3214072b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3214068480 msgr2=0x7f321406a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3214068480 0x7f321406a940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f322c01a0d0 tx=0x7f322c0a6040 comp rx=0 tx=0).stop 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230144bf0 msgr2=0x7f3230142880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 --2- 172.21.15.12:0/1697048584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3230144bf0 0x7f3230142880 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3228007d90 tx=0x7f322800e4b0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f3236d50640 1 -- 172.21.15.12:0/1697048584 reap_dead start 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 shutdown_connections 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 >> 172.21.15.12:0/1697048584 conn(0x7f32300b9340 msgr2=0x7f323013d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 shutdown_connections 2024-04-27T07:04:25.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.797+0000 7f32387b3640 1 -- 172.21.15.12:0/1697048584 wait complete. 2024-04-27T07:04:25.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 --> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78e4069330 con 0x7f78c8070520 2024-04-27T07:04:25.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78d67fc640 1 -- 172.21.15.12:0/155423390 <== osd.1 v2:172.21.15.12:6800/3260695366 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f78e4069330 con 0x7f78c8070520 2024-04-27T07:04:25.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 >> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] conn(0x7f78c8070520 msgr2=0x7f78c8072940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] conn(0x7f78c8070520 0x7f78c8072940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f78c8068380 msgr2=0x7f78c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f78c8068380 0x7f78c806a840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78d80ad000 tx=0x7f78d8008040 comp rx=0 tx=0).stop 2024-04-27T07:04:25.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e4141870 msgr2=0x7f78e41e0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 --2- 172.21.15.12:0/155423390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e4141870 0x7f78e41e0bc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78e000d5f0 tx=0x7f78e000dab0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78ea3e8640 1 -- 172.21.15.12:0/155423390 reap_dead start 2024-04-27T07:04:25.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 shutdown_connections 2024-04-27T07:04:25.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 >> 172.21.15.12:0/155423390 conn(0x7f78e4068190 msgr2=0x7f78e40b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 shutdown_connections 2024-04-27T07:04:25.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f78eabe9640 1 -- 172.21.15.12:0/155423390 wait complete. 2024-04-27T07:04:25.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f2f110dc640 1 Processor -- start 2024-04-27T07:04:25.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.801+0000 7f2f110dc640 1 -- start start 2024-04-27T07:04:25.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 --> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a14137060 con 0x7f49f4070650 2024-04-27T07:04:25.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a02ffd640 1 -- 172.21.15.12:0/3900620592 <== osd.6 v2:172.21.15.53:6808/289427539 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a14137060 con 0x7f49f4070650 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] conn(0x7f49f4070650 msgr2=0x7f49f4072a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] conn(0x7f49f4070650 0x7f49f4072a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f49f40683d0 msgr2=0x7f49f406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f49f40683d0 0x7f49f406a890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a040b2000 tx=0x7f4a0400b040 comp rx=0 tx=0).stop 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a141440f0 msgr2=0x7f4a14138ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 --2- 172.21.15.12:0/3900620592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a141440f0 0x7f4a14138ca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00b410 tx=0x7f4a0c00b8d0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a12d9d640 1 -- 172.21.15.12:0/3900620592 reap_dead start 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 shutdown_connections 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 >> 172.21.15.12:0/3900620592 conn(0x7f4a140b9310 msgr2=0x7f4a140b6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 shutdown_connections 2024-04-27T07:04:25.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f4a1359e640 1 -- 172.21.15.12:0/3900620592 wait complete. 2024-04-27T07:04:25.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f110dc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2f04037340 0x7f2f04039770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f110dc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f04038e10 0x7f2f040cab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f110dc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f04037720 con 0x7f2f040c40b0 2024-04-27T07:04:25.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f110dc640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f2f04039dd0 con 0x7f2f04038e10 2024-04-27T07:04:25.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f110dc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f2f04037d20 con 0x7f2f04037340 2024-04-27T07:04:25.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f0b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f04038e10 0x7f2f040cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f0b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f04038e10 0x7f2f040cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42188/0 (socket says 172.21.15.12:42188) 2024-04-27T07:04:25.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f0b7fe640 1 -- 172.21.15.12:0/919806079 learned_addr learned my addr 172.21.15.12:0/919806079 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f0bfff640 1 --2- 172.21.15.12:0/919806079 >> v2:172.21.15.53:3300/0 conn(0x7f2f04037340 0x7f2f04039770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f0b7fe640 1 -- 172.21.15.12:0/919806079 >> v2:172.21.15.53:3300/0 conn(0x7f2f04037340 msgr2=0x7f2f04039770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:25.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f0b7fe640 1 --2- 172.21.15.12:0/919806079 >> v2:172.21.15.53:3300/0 conn(0x7f2f04037340 0x7f2f04039770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f0b7fe640 1 -- 172.21.15.12:0/919806079 >> v1:172.21.15.53:6789/0 conn(0x7f2f040c40b0 legacy=0x7f2f040c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.805+0000 7f2f0b7fe640 1 -- 172.21.15.12:0/919806079 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f040ca5f0 con 0x7f2f04038e10 2024-04-27T07:04:25.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f0b7fe640 1 --2- 172.21.15.12:0/919806079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f04038e10 0x7f2f040cab10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f0403bf00 tx=0x7f2ef801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7970beb9cc66234 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 --> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1340be630 con 0x7fa1200706d0 2024-04-27T07:04:25.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f0affd640 1 -- 172.21.15.12:0/919806079 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80b0070 con 0x7f2f04038e10 2024-04-27T07:04:25.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1397fa640 1 -- 172.21.15.12:0/1675495849 <== osd.0 v2:172.21.15.12:6813/3217679893 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa1340be630 con 0x7fa1200706d0 2024-04-27T07:04:25.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 >> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] conn(0x7fa1200706d0 msgr2=0x7fa120072af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] conn(0x7fa1200706d0 0x7fa120072af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f0affd640 1 -- 172.21.15.12:0/919806079 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef80ab600 con 0x7f2f04038e10 2024-04-27T07:04:25.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f0affd640 1 -- 172.21.15.12:0/919806079 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80ab9d0 con 0x7f2f04038e10 2024-04-27T07:04:25.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f110dc640 1 -- 172.21.15.12:0/919806079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f04038e10 msgr2=0x7f2f040cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f110dc640 1 --2- 172.21.15.12:0/919806079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f04038e10 0x7f2f040cab10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f0403bf00 tx=0x7f2ef801c420 comp rx=0 tx=0).stop 2024-04-27T07:04:25.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f110dc640 1 -- 172.21.15.12:0/919806079 shutdown_connections 2024-04-27T07:04:25.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f110dc640 1 --2- 172.21.15.12:0/919806079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f04038e10 0x7f2f040cab10 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f0403bf00 tx=0x7f2ef801c420 comp rx=0 tx=0).stop 2024-04-27T07:04:25.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f110dc640 1 --2- 172.21.15.12:0/919806079 >> v2:172.21.15.53:3300/0 conn(0x7f2f04037340 0x7f2f04039770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f110dc640 1 -- 172.21.15.12:0/919806079 >> 172.21.15.12:0/919806079 conn(0x7f2f0403c6b0 msgr2=0x7f2f0403cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f110dc640 1 -- 172.21.15.12:0/919806079 shutdown_connections 2024-04-27T07:04:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa120068450 msgr2=0x7fa12006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa120068450 0x7fa12006a910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1300099f0 tx=0x7fa130009190 comp rx=0 tx=0).stop 2024-04-27T07:04:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1340370b0 msgr2=0x7fa1340bef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 --2- 172.21.15.12:0/1675495849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1340370b0 0x7fa1340bef10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1280af040 tx=0x7fa1280190c0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1411cd640 1 -- 172.21.15.12:0/1675495849 reap_dead start 2024-04-27T07:04:25.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 shutdown_connections 2024-04-27T07:04:25.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 >> 172.21.15.12:0/1675495849 conn(0x7fa13403a890 msgr2=0x7fa134004870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 shutdown_connections 2024-04-27T07:04:25.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7fa1419ce640 1 -- 172.21.15.12:0/1675495849 wait complete. 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.809+0000 7f2f110dc640 1 -- 172.21.15.12:0/919806079 wait complete. 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f110dc640 1 Processor -- start 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f110dc640 1 -- start start 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f110dc640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2f04037340 0x7f2f04036430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f110dc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040c40b0 0x7f2f04175330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f110dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f040cb4a0 0x7f2f0416ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f110dc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040d7480 con 0x7f2f040c40b0 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f110dc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040d7600 con 0x7f2f040cb4a0 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f110dc640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f2f040d7780 con 0x7f2f04037340 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f108db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f040cb4a0 0x7f2f0416ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f108db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f040cb4a0 0x7f2f0416ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:37192/0 (socket says 172.21.15.12:37192) 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f0b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040c40b0 0x7f2f04175330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f108db640 1 -- 172.21.15.12:0/1888276549 learned_addr learned my addr 172.21.15.12:0/1888276549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:25.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f0bfff640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2f04037340 0x7f2f04036430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f0b7fe640 1 -- 172.21.15.12:0/1888276549 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2f04037340 msgr2=0x7f2f04036430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f0b7fe640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2f04037340 0x7f2f04036430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f0b7fe640 1 -- 172.21.15.12:0/1888276549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f040cb4a0 msgr2=0x7f2f0416ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f0b7fe640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f040cb4a0 0x7f2f0416ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f0b7fe640 1 -- 172.21.15.12:0/1888276549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f040ccfa0 con 0x7f2f040c40b0 2024-04-27T07:04:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.813+0000 7f2f0b7fe640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040c40b0 0x7f2f04175330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80ab000 tx=0x7f2ef80191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f804da5b640 1 -- 172.21.15.12:0/3890030062 --> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f80400ca4c0 con 0x7f802c0705b0 2024-04-27T07:04:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f2f08ff9640 1 -- 172.21.15.12:0/1888276549 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80b0050 con 0x7f2f040c40b0 2024-04-27T07:04:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f040c6930 con 0x7f2f040c40b0 2024-04-27T07:04:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f040c5d30 con 0x7f2f040c40b0 2024-04-27T07:04:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f2f08ff9640 1 -- 172.21.15.12:0/1888276549 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef80ba070 con 0x7f2f040c40b0 2024-04-27T07:04:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 --> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd680069330 con 0x7fd6640706a0 2024-04-27T07:04:25.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f4480e5a640 1 -- 172.21.15.12:0/1064336048 --> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f447c1e7ee0 con 0x7f445c070630 2024-04-27T07:04:25.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f2f08ff9640 1 -- 172.21.15.12:0/1888276549 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80b47d0 con 0x7f2f040c40b0 2024-04-27T07:04:25.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f80457fa640 1 -- 172.21.15.12:0/3890030062 <== osd.5 v2:172.21.15.53:6824/2061790692 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f80400ca4c0 con 0x7f802c0705b0 2024-04-27T07:04:25.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7fd6767fc640 1 -- 172.21.15.12:0/310315404 <== osd.2 v2:172.21.15.12:6811/3457925485 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd680069330 con 0x7fd6640706a0 2024-04-27T07:04:25.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2eec001040 con 0x7f2f040c40b0 2024-04-27T07:04:25.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f446affd640 1 -- 172.21.15.12:0/1064336048 <== osd.7 v2:172.21.15.53:6816/2338388162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f447c1e7ee0 con 0x7f445c070630 2024-04-27T07:04:25.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f8032ffd640 1 -- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] conn(0x7f802c0705b0 msgr2=0x7f802c0729d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f8032ffd640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] conn(0x7f802c0705b0 0x7f802c0729d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f8032ffd640 1 -- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f802c0683b0 msgr2=0x7f802c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f8032ffd640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f802c0683b0 0x7f802c06a870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f803c007fd0 tx=0x7f803c007940 comp rx=0 tx=0).stop 2024-04-27T07:04:25.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f8032ffd640 1 -- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80400369a0 msgr2=0x7f80400c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f8032ffd640 1 --2- 172.21.15.12:0/3890030062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80400369a0 0x7f80400c6350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f803400b410 tx=0x7f803400b8d0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f804d25a640 1 -- 172.21.15.12:0/3890030062 reap_dead start 2024-04-27T07:04:25.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f8032ffd640 1 -- 172.21.15.12:0/3890030062 shutdown_connections 2024-04-27T07:04:25.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f8032ffd640 1 -- 172.21.15.12:0/3890030062 >> 172.21.15.12:0/3890030062 conn(0x7f804003a890 msgr2=0x7f8040004870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f8032ffd640 1 -- 172.21.15.12:0/3890030062 shutdown_connections 2024-04-27T07:04:25.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f8032ffd640 1 -- 172.21.15.12:0/3890030062 wait complete. 2024-04-27T07:04:25.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f2f08ff9640 1 -- 172.21.15.12:0/1888276549 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2ef80b4970 con 0x7f2f040c40b0 2024-04-27T07:04:25.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f4468ff9640 1 -- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] conn(0x7f445c070630 msgr2=0x7f445c072a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 >> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] conn(0x7fd6640706a0 msgr2=0x7fd664072ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7fd68808d640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] conn(0x7fd6640706a0 0x7fd664072ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.821+0000 7f4468ff9640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] conn(0x7f445c070630 0x7f445c072a50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f4468ff9640 1 -- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f445c0683b0 msgr2=0x7f445c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f4468ff9640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f445c0683b0 0x7f445c06a870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f447c1453e0 tx=0x7f446c008040 comp rx=0 tx=0).stop 2024-04-27T07:04:25.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f4468ff9640 1 -- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f447c09acd0 msgr2=0x7f447c0711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f4468ff9640 1 --2- 172.21.15.12:0/1064336048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f447c09acd0 0x7f447c0711b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f447400cb20 tx=0x7f44740020d0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd664068480 msgr2=0x7fd66406a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7fd68808d640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd664068480 0x7fd66406a940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6700045a0 tx=0x7fd670020040 comp rx=0 tx=0).stop 2024-04-27T07:04:25.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68014f820 msgr2=0x7fd6801487d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7fd68808d640 1 --2- 172.21.15.12:0/310315404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd68014f820 0x7fd6801487d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd6800ba4c0 tx=0x7fd66c005840 comp rx=0 tx=0).stop 2024-04-27T07:04:25.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f447ad9d640 1 -- 172.21.15.12:0/1064336048 reap_dead start 2024-04-27T07:04:25.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f4468ff9640 1 -- 172.21.15.12:0/1064336048 shutdown_connections 2024-04-27T07:04:25.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f4468ff9640 1 -- 172.21.15.12:0/1064336048 >> 172.21.15.12:0/1064336048 conn(0x7f447c0b5e40 msgr2=0x7f447c151990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7fd68662a640 1 -- 172.21.15.12:0/310315404 reap_dead start 2024-04-27T07:04:25.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 shutdown_connections 2024-04-27T07:04:25.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 >> 172.21.15.12:0/310315404 conn(0x7fd6800b6ff0 msgr2=0x7fd6800b82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 shutdown_connections 2024-04-27T07:04:25.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f4468ff9640 1 -- 172.21.15.12:0/1064336048 shutdown_connections 2024-04-27T07:04:25.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f4468ff9640 1 -- 172.21.15.12:0/1064336048 wait complete. 2024-04-27T07:04:25.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7fd68808d640 1 -- 172.21.15.12:0/310315404 wait complete. 2024-04-27T07:04:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f2f08ff9640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2ef0068450 0x7f2ef006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f2f08ff9640 1 -- 172.21.15.12:0/1888276549 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ef8120560 con 0x7f2f040c40b0 2024-04-27T07:04:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f2f0bfff640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2ef0068450 0x7f2ef006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f2f08ff9640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] conn(0x7f2ef00706d0 0x7f2ef0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f2f08ff9640 1 -- 172.21.15.12:0/1888276549 --> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ef80a8ec0 con 0x7f2ef00706d0 2024-04-27T07:04:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f2f08ff9640 1 -- 172.21.15.12:0/1888276549 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2ef8127050 con 0x7f2f040c40b0 2024-04-27T07:04:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f2f108db640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] conn(0x7f2ef00706d0 0x7f2ef0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f2f108db640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] conn(0x7f2ef00706d0 0x7f2ef0072af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f2f0bfff640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2ef0068450 0x7f2ef006a910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009b60 tx=0x7f2f00009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:25.853 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:25.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-27T07:04:25.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.825+0000 7f2f08ff9640 1 -- 172.21.15.12:0/1888276549 <== osd.3 v2:172.21.15.12:6808/2279477907 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ef80a8ec0 con 0x7f2ef00706d0 2024-04-27T07:04:25.854 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:25.855 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T07:04:25.855 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:25.856 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T07:04:25.856 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:25.857 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-27T07:04:25.857 INFO:teuthology.orchestra.run.smithi012.stdout:55834574856 2024-04-27T07:04:25.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-27T07:04:25.859 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:25.859 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-27T07:04:25.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 --> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2eec002de0 con 0x7f2ef00706d0 2024-04-27T07:04:25.861 INFO:teuthology.orchestra.run.smithi012.stdout:55834574853 2024-04-27T07:04:25.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T07:04:25.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f08ff9640 1 -- 172.21.15.12:0/1888276549 <== osd.3 v2:172.21.15.12:6808/2279477907 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2eec002de0 con 0x7f2ef00706d0 2024-04-27T07:04:25.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 >> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] conn(0x7f2ef00706d0 msgr2=0x7f2ef0072af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] conn(0x7f2ef00706d0 0x7f2ef0072af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2ef0068450 msgr2=0x7f2ef006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2ef0068450 0x7f2ef006a910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009b60 tx=0x7f2f00009190 comp rx=0 tx=0).stop 2024-04-27T07:04:25.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040c40b0 msgr2=0x7f2f04175330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:25.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 --2- 172.21.15.12:0/1888276549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f040c40b0 0x7f2f04175330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80ab000 tx=0x7f2ef80191c0 comp rx=0 tx=0).stop 2024-04-27T07:04:25.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f108db640 1 -- 172.21.15.12:0/1888276549 reap_dead start 2024-04-27T07:04:25.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 shutdown_connections 2024-04-27T07:04:25.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 >> 172.21.15.12:0/1888276549 conn(0x7f2f0403c6b0 msgr2=0x7f2f040c2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:25.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 shutdown_connections 2024-04-27T07:04:25.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:25.853+0000 7f2f110dc640 1 -- 172.21.15.12:0/1888276549 wait complete. 2024-04-27T07:04:25.885 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:25.885 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T07:04:26.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704748f640 1 Processor -- start 2024-04-27T07:04:26.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704748f640 1 -- start start 2024-04-27T07:04:26.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704748f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7040147420 0x7f70400af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704748f640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f704009aa70 0x7f704009aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704748f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7040147800 con 0x7f70400b51a0 2024-04-27T07:04:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704748f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f70400af950 con 0x7f7040147420 2024-04-27T07:04:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704748f640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f70400afb70 con 0x7f704009aa70 2024-04-27T07:04:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704522b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7040147420 0x7f70400af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704522b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7040147420 0x7f70400af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42206/0 (socket says 172.21.15.12:42206) 2024-04-27T07:04:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704522b640 1 -- 172.21.15.12:0/2280285213 learned_addr learned my addr 172.21.15.12:0/2280285213 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f7044a2a640 1 --2- 172.21.15.12:0/2280285213 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f704009aa70 0x7f704009aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704522b640 1 -- 172.21.15.12:0/2280285213 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f704009aa70 msgr2=0x7f704009aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704522b640 1 --2- 172.21.15.12:0/2280285213 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f704009aa70 0x7f704009aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f704522b640 1 -- 172.21.15.12:0/2280285213 >> v1:172.21.15.53:6789/0 conn(0x7f70400b51a0 legacy=0x7f70400b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.005+0000 7f7037fff640 1 -- 172.21.15.12:0/2280285213 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 620401554 0 0) 0x7f7040147800 con 0x7f70400b51a0 2024-04-27T07:04:26.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.009+0000 7f704522b640 1 -- 172.21.15.12:0/2280285213 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70400afe50 con 0x7f7040147420 2024-04-27T07:04:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.009+0000 7f704522b640 1 --2- 172.21.15.12:0/2280285213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7040147420 0x7f70400af410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7030009870 tx=0x7f703001e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac59e48a51f1ad84 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.009+0000 7f7037fff640 1 -- 172.21.15.12:0/2280285213 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300b4070 con 0x7f7040147420 2024-04-27T07:04:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.009+0000 7f7037fff640 1 -- 172.21.15.12:0/2280285213 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70300a7860 con 0x7f7040147420 2024-04-27T07:04:26.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.009+0000 7f7037fff640 1 -- 172.21.15.12:0/2280285213 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300a7c30 con 0x7f7040147420 2024-04-27T07:04:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 -- 172.21.15.12:0/2280285213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7040147420 msgr2=0x7f70400af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 --2- 172.21.15.12:0/2280285213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7040147420 0x7f70400af410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7030009870 tx=0x7f703001e990 comp rx=0 tx=0).stop 2024-04-27T07:04:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 -- 172.21.15.12:0/2280285213 shutdown_connections 2024-04-27T07:04:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 --2- 172.21.15.12:0/2280285213 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f704009aa70 0x7f704009aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 --2- 172.21.15.12:0/2280285213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7040147420 0x7f70400af410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 -- 172.21.15.12:0/2280285213 >> 172.21.15.12:0/2280285213 conn(0x7f70400b5e40 msgr2=0x7f70400b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 -- 172.21.15.12:0/2280285213 shutdown_connections 2024-04-27T07:04:26.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 -- 172.21.15.12:0/2280285213 wait complete. 2024-04-27T07:04:26.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 Processor -- start 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 -- start start 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f704009aa70 0x7f704013c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70400b51a0 0x7f7040136800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040147420 0x7f7040141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7040150f20 con 0x7f70400b51a0 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f70401510a0 con 0x7f7040147420 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704748f640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7040151220 con 0x7f704009aa70 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f7044a2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70400b51a0 0x7f7040136800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f7044a2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70400b51a0 0x7f7040136800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42212/0 (socket says 172.21.15.12:42212) 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f7044a2a640 1 -- 172.21.15.12:0/3289521975 learned_addr learned my addr 172.21.15.12:0/3289521975 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f7045a2c640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040147420 0x7f7040141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.013+0000 7f704522b640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f704009aa70 0x7f704013c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.017+0000 7f7044a2a640 1 -- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f704009aa70 msgr2=0x7f704013c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.017+0000 7f7044a2a640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f704009aa70 0x7f704013c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.017+0000 7f7044a2a640 1 -- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040147420 msgr2=0x7f7040141ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.017+0000 7f7044a2a640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040147420 0x7f7040141ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.017+0000 7f7044a2a640 1 -- 172.21.15.12:0/3289521975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70401e8390 con 0x7f70400b51a0 2024-04-27T07:04:26.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.017+0000 7f7044a2a640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70400b51a0 0x7f7040136800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f703800e3b0 tx=0x7f703800e870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.017+0000 7f7035ffb640 1 -- 172.21.15.12:0/3289521975 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703800a0d0 con 0x7f70400b51a0 2024-04-27T07:04:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.017+0000 7f7035ffb640 1 -- 172.21.15.12:0/3289521975 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7038019070 con 0x7f70400b51a0 2024-04-27T07:04:26.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.017+0000 7f704748f640 1 -- 172.21.15.12:0/3289521975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70401e8670 con 0x7f70400b51a0 2024-04-27T07:04:26.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.017+0000 7f7035ffb640 1 -- 172.21.15.12:0/3289521975 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703800a4d0 con 0x7f70400b51a0 2024-04-27T07:04:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.021+0000 7f704748f640 1 -- 172.21.15.12:0/3289521975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704013d340 con 0x7f70400b51a0 2024-04-27T07:04:26.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.021+0000 7f7035ffb640 1 -- 172.21.15.12:0/3289521975 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f703800a670 con 0x7f70400b51a0 2024-04-27T07:04:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.021+0000 7f7035ffb640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70240683d0 0x7f702406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.021+0000 7f7035ffb640 1 -- 172.21.15.12:0/3289521975 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f703808b170 con 0x7f70400b51a0 2024-04-27T07:04:26.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.021+0000 7f704522b640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70240683d0 0x7f702406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.025+0000 7f704522b640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70240683d0 0x7f702406a890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f70300b1000 tx=0x7f7030009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.025+0000 7f704748f640 1 -- 172.21.15.12:0/3289521975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7040068c20 con 0x7f70400b51a0 2024-04-27T07:04:26.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.025+0000 7f3422bca640 1 Processor -- start 2024-04-27T07:04:26.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.025+0000 7f3422bca640 1 -- start start 2024-04-27T07:04:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3422bca640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 0x7f341c0b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3422bca640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f341c0b9ac0 0x7f341c1472d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3422bca640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f341c06e840 con 0x7f341c06e3c0 2024-04-27T07:04:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3422bca640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f341c13e520 con 0x7f341c0b9ac0 2024-04-27T07:04:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3422bca640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f341c0b5b70 con 0x7f341c0b4720 2024-04-27T07:04:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3421167640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f341c06e3c0 0x7f341c0afd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40880/0 (socket says 172.21.15.12:40880) 2024-04-27T07:04:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3421167640 1 -- 172.21.15.12:0/2649018049 learned_addr learned my addr 172.21.15.12:0/2649018049 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:26.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1c821640 1 Processor -- start 2024-04-27T07:04:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1c821640 1 -- start start 2024-04-27T07:04:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3420966640 1 --2- 172.21.15.12:0/2649018049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 0x7f341c0b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f341bfff640 1 --2- 172.21.15.12:0/2649018049 >> v2:172.21.15.53:3300/0 conn(0x7f341c0b9ac0 0x7f341c1472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f6516422640 1 Processor -- start 2024-04-27T07:04:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f6516422640 1 -- start start 2024-04-27T07:04:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1c821640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8f180b82c0 0x7f8f180b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1c821640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f181484e0 0x7f8f180b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1c821640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f1806c1c0 con 0x7f8f1806bde0 2024-04-27T07:04:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1c821640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f8f180b6d80 con 0x7f8f180b82c0 2024-04-27T07:04:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1c821640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8f180b87a0 con 0x7f8f181484e0 2024-04-27T07:04:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f16d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8f1806bde0 0x7f8f18147dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40896/0 (socket says 172.21.15.12:40896) 2024-04-27T07:04:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f16d9d640 1 -- 172.21.15.12:0/1911644155 learned_addr learned my addr 172.21.15.12:0/1911644155 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:26.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1659c640 1 --2- 172.21.15.12:0/1911644155 >> v2:172.21.15.12:3300/0 conn(0x7f8f180b82c0 0x7f8f180b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f6516422640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f65100baf60 0x7f65100b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f6516422640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f65100adee0 0x7f651006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f6516422640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65100b5fb0 con 0x7f651006bb80 2024-04-27T07:04:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f6516422640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f651006fb70 con 0x7f65100baf60 2024-04-27T07:04:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f6516422640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f65100b6900 con 0x7f65100adee0 2024-04-27T07:04:26.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f65149bf640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f651006bb80 0x7f651006bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:54850/0 (socket says 172.21.15.12:54850) 2024-04-27T07:04:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f65149bf640 1 -- 172.21.15.12:0/2828209542 learned_addr learned my addr 172.21.15.12:0/2828209542 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f650f7fe640 1 --2- 172.21.15.12:0/2828209542 >> v2:172.21.15.53:3300/0 conn(0x7f65100adee0 0x7f651006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f650ffff640 1 --2- 172.21.15.12:0/2828209542 >> v2:172.21.15.12:3300/0 conn(0x7f65100baf60 0x7f65100b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f650ffff640 1 -- 172.21.15.12:0/2828209542 >> v2:172.21.15.53:3300/0 conn(0x7f65100adee0 msgr2=0x7f651006e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f650ffff640 1 --2- 172.21.15.12:0/2828209542 >> v2:172.21.15.53:3300/0 conn(0x7f65100adee0 0x7f651006e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f650ffff640 1 -- 172.21.15.12:0/2828209542 >> v1:172.21.15.53:6789/0 conn(0x7f651006bb80 legacy=0x7f651006bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f650ffff640 1 -- 172.21.15.12:0/2828209542 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65100bb340 con 0x7f65100baf60 2024-04-27T07:04:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f650f7fe640 1 --2- 172.21.15.12:0/2828209542 >> v2:172.21.15.53:3300/0 conn(0x7f65100adee0 0x7f651006e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:26.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3420966640 1 -- 172.21.15.12:0/2649018049 >> v2:172.21.15.53:3300/0 conn(0x7f341c0b9ac0 msgr2=0x7f341c1472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3420966640 1 --2- 172.21.15.12:0/2649018049 >> v2:172.21.15.53:3300/0 conn(0x7f341c0b9ac0 0x7f341c1472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3420966640 1 -- 172.21.15.12:0/2649018049 >> v1:172.21.15.12:6789/0 conn(0x7f341c06e3c0 legacy=0x7f341c0afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f3420966640 1 -- 172.21.15.12:0/2649018049 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341c0b4c00 con 0x7f341c0b4720 2024-04-27T07:04:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1659c640 1 -- 172.21.15.12:0/1911644155 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f181484e0 msgr2=0x7f8f180b6510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-27T07:04:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1659c640 1 --2- 172.21.15.12:0/1911644155 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f181484e0 0x7f8f180b6510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1659c640 1 -- 172.21.15.12:0/1911644155 >> v1:172.21.15.12:6789/0 conn(0x7f8f1806bde0 legacy=0x7f8f18147dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.029+0000 7f8f1659c640 1 -- 172.21.15.12:0/1911644155 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f180b8a80 con 0x7f8f180b82c0 2024-04-27T07:04:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f3420966640 1 --2- 172.21.15.12:0/2649018049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 0x7f341c0b62e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3414009d90 tx=0x7f341401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32fb8b90f5224282 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f7035ffb640 1 -- 172.21.15.12:0/3289521975 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7038057330 con 0x7f70400b51a0 2024-04-27T07:04:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f341b7fe640 1 -- 172.21.15.12:0/2649018049 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140b2070 con 0x7f341c0b4720 2024-04-27T07:04:26.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f341b7fe640 1 -- 172.21.15.12:0/2649018049 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34140a57b0 con 0x7f341c0b4720 2024-04-27T07:04:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f341b7fe640 1 -- 172.21.15.12:0/2649018049 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140a5b80 con 0x7f341c0b4720 2024-04-27T07:04:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f650ffff640 1 --2- 172.21.15.12:0/2828209542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100baf60 0x7f65100b4ef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f65100714f0 tx=0x7f650801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b669d580e49620 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f650effd640 1 -- 172.21.15.12:0/2828209542 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65080b2070 con 0x7f65100baf60 2024-04-27T07:04:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f8f1659c640 1 --2- 172.21.15.12:0/1911644155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f180b82c0 0x7f8f180b5fd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0097a0 tx=0x7f8f0c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730b2376745fa201 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f8f15d9b640 1 -- 172.21.15.12:0/1911644155 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0aa600 con 0x7f8f180b82c0 2024-04-27T07:04:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f8f15d9b640 1 -- 172.21.15.12:0/1911644155 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0af070 con 0x7f8f180b82c0 2024-04-27T07:04:26.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f8f15d9b640 1 -- 172.21.15.12:0/1911644155 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0aa9c0 con 0x7f8f180b82c0 2024-04-27T07:04:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f3422bca640 1 -- 172.21.15.12:0/2649018049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 msgr2=0x7f341c0b62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f3422bca640 1 --2- 172.21.15.12:0/2649018049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 0x7f341c0b62e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3414009d90 tx=0x7f341401c8e0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f3422bca640 1 -- 172.21.15.12:0/2649018049 shutdown_connections 2024-04-27T07:04:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f3422bca640 1 --2- 172.21.15.12:0/2649018049 >> v2:172.21.15.53:3300/0 conn(0x7f341c0b9ac0 0x7f341c1472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f3422bca640 1 --2- 172.21.15.12:0/2649018049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 0x7f341c0b62e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f3422bca640 1 -- 172.21.15.12:0/2649018049 >> 172.21.15.12:0/2649018049 conn(0x7f341c09aa70 msgr2=0x7f341c0b93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f3422bca640 1 -- 172.21.15.12:0/2649018049 shutdown_connections 2024-04-27T07:04:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f650effd640 1 -- 172.21.15.12:0/2828209542 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65080a5710 con 0x7f65100baf60 2024-04-27T07:04:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 -- 172.21.15.12:0/2828209542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100baf60 msgr2=0x7f65100b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 -- 172.21.15.12:0/1911644155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f180b82c0 msgr2=0x7f8f180b5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 --2- 172.21.15.12:0/1911644155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f180b82c0 0x7f8f180b5fd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0097a0 tx=0x7f8f0c01c4c0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 --2- 172.21.15.12:0/2828209542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100baf60 0x7f65100b4ef0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f65100714f0 tx=0x7f650801c730 comp rx=0 tx=0).stop 2024-04-27T07:04:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 -- 172.21.15.12:0/2828209542 shutdown_connections 2024-04-27T07:04:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 --2- 172.21.15.12:0/2828209542 >> v2:172.21.15.53:3300/0 conn(0x7f65100adee0 0x7f651006e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 --2- 172.21.15.12:0/2828209542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100baf60 0x7f65100b4ef0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 -- 172.21.15.12:0/2828209542 >> 172.21.15.12:0/2828209542 conn(0x7f6510067f80 msgr2=0x7f65100ba5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 -- 172.21.15.12:0/1911644155 shutdown_connections 2024-04-27T07:04:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 --2- 172.21.15.12:0/1911644155 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f181484e0 0x7f8f180b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 --2- 172.21.15.12:0/1911644155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f180b82c0 0x7f8f180b5fd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 -- 172.21.15.12:0/1911644155 >> 172.21.15.12:0/1911644155 conn(0x7f8f1809c270 msgr2=0x7f8f1809c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.033+0000 7f3422bca640 1 -- 172.21.15.12:0/2649018049 wait complete. 2024-04-27T07:04:26.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3422bca640 1 Processor -- start 2024-04-27T07:04:26.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 -- 172.21.15.12:0/1911644155 shutdown_connections 2024-04-27T07:04:26.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 -- 172.21.15.12:0/1911644155 wait complete. 2024-04-27T07:04:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3422bca640 1 -- start start 2024-04-27T07:04:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3422bca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f341c06e3c0 0x7f341c1ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3422bca640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 0x7f341c072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3422bca640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f341c0b9ac0 0x7f341c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3422bca640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f341c149170 con 0x7f341c0b9ac0 2024-04-27T07:04:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3422bca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f341c148ff0 con 0x7f341c06e3c0 2024-04-27T07:04:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3422bca640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f341c1492f0 con 0x7f341c0b4720 2024-04-27T07:04:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3421167640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f341c0b9ac0 0x7f341c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3420966640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f341c06e3c0 0x7f341c1ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f341bfff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 0x7f341c072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3421167640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f341c0b9ac0 0x7f341c143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42234/0 (socket says 172.21.15.12:42234) 2024-04-27T07:04:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 Processor -- start 2024-04-27T07:04:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f341bfff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 0x7f341c072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:58654/0 (socket says 172.21.15.12:58654) 2024-04-27T07:04:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3420966640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f341c06e3c0 0x7f341c1ed6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:37214/0 (socket says 172.21.15.12:37214) 2024-04-27T07:04:26.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f341bfff640 1 -- 172.21.15.12:0/270295322 learned_addr learned my addr 172.21.15.12:0/270295322 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 -- start start 2024-04-27T07:04:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 -- 172.21.15.12:0/2828209542 shutdown_connections 2024-04-27T07:04:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 -- 172.21.15.12:0/2828209542 wait complete. 2024-04-27T07:04:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 Processor -- start 2024-04-27T07:04:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 -- start start 2024-04-27T07:04:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f651006bb80 0x7f65100b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65100adee0 0x7f6510099570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100982a0 0x7f6510098710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3421167640 1 -- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 msgr2=0x7f341c072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3421167640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 0x7f341c072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3421167640 1 -- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f341c06e3c0 msgr2=0x7f341c1ed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3421167640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f341c06e3c0 0x7f341c1ed6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3421167640 1 -- 172.21.15.12:0/270295322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341c073dd0 con 0x7f341c0b9ac0 2024-04-27T07:04:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f651006eb90 con 0x7f65100982a0 2024-04-27T07:04:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f651006ea10 con 0x7f651006bb80 2024-04-27T07:04:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f6516422640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f6510159720 con 0x7f65100adee0 2024-04-27T07:04:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f65149bf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100982a0 0x7f6510098710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f65149bf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100982a0 0x7f6510098710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42240/0 (socket says 172.21.15.12:42240) 2024-04-27T07:04:26.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f65149bf640 1 -- 172.21.15.12:0/1373321917 learned_addr learned my addr 172.21.15.12:0/1373321917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f3421167640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f341c0b9ac0 0x7f341c143d30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c520 tx=0x7f340c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f65149bf640 1 -- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65100adee0 msgr2=0x7f6510099570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-27T07:04:26.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f1806bde0 0x7f8f18091110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f180b82c0 0x7f8f181e0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f181484e0 0x7f8f181e1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f18154ec0 con 0x7f8f181484e0 2024-04-27T07:04:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f18154d40 con 0x7f8f180b82c0 2024-04-27T07:04:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f8f1c821640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8f18155040 con 0x7f8f1806bde0 2024-04-27T07:04:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f65149bf640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65100adee0 0x7f6510099570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f16d9d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f1806bde0 0x7f8f18091110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f65149bf640 1 -- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f651006bb80 msgr2=0x7f65100b05d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-27T07:04:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f65149bf640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f651006bb80 0x7f65100b05d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.037+0000 7f65149bf640 1 -- 172.21.15.12:0/1373321917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6510099c90 con 0x7f65100982a0 2024-04-27T07:04:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f16d9d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f1806bde0 0x7f8f18091110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:58660/0 (socket says 172.21.15.12:58660) 2024-04-27T07:04:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f16d9d640 1 -- 172.21.15.12:0/2469211915 learned_addr learned my addr 172.21.15.12:0/2469211915 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f1759e640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f181484e0 0x7f8f181e1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f1659c640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f180b82c0 0x7f8f181e0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f65149bf640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100982a0 0x7f6510098710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f650000c780 tx=0x7f650000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f650cff9640 1 -- 172.21.15.12:0/1373321917 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500018070 con 0x7f65100982a0 2024-04-27T07:04:26.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f1759e640 1 -- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f1806bde0 msgr2=0x7f8f18091110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f1759e640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f1806bde0 0x7f8f18091110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f1759e640 1 -- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f180b82c0 msgr2=0x7f8f181e0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f1759e640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f180b82c0 0x7f8f181e0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f1759e640 1 -- 172.21.15.12:0/2469211915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f18092b40 con 0x7f8f181484e0 2024-04-27T07:04:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f650cff9640 1 -- 172.21.15.12:0/1373321917 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65000072d0 con 0x7f65100982a0 2024-04-27T07:04:26.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f34197fa640 1 -- 172.21.15.12:0/270295322 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c017070 con 0x7f341c0b9ac0 2024-04-27T07:04:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f650cff9640 1 -- 172.21.15.12:0/1373321917 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65000076a0 con 0x7f65100982a0 2024-04-27T07:04:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f34197fa640 1 -- 172.21.15.12:0/270295322 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340c005ac0 con 0x7f341c0b9ac0 2024-04-27T07:04:26.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f1759e640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f181484e0 0x7f8f181e1000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800b3e0 tx=0x7f8f0800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f3422bca640 1 -- 172.21.15.12:0/270295322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341c071450 con 0x7f341c0b9ac0 2024-04-27T07:04:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f34197fa640 1 -- 172.21.15.12:0/270295322 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c0070f0 con 0x7f341c0b9ac0 2024-04-27T07:04:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f3422bca640 1 -- 172.21.15.12:0/270295322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341c072740 con 0x7f341c0b9ac0 2024-04-27T07:04:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f6516422640 1 -- 172.21.15.12:0/1373321917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6510097d80 con 0x7f65100982a0 2024-04-27T07:04:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f6516422640 1 -- 172.21.15.12:0/1373321917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65100b1010 con 0x7f65100982a0 2024-04-27T07:04:26.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f3422bca640 1 -- 172.21.15.12:0/270295322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341c068c20 con 0x7f341c0b9ac0 2024-04-27T07:04:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8eff7fe640 1 -- 172.21.15.12:0/2469211915 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08015020 con 0x7f8f181484e0 2024-04-27T07:04:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8eff7fe640 1 -- 172.21.15.12:0/2469211915 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f08011830 con 0x7f8f181484e0 2024-04-27T07:04:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8eff7fe640 1 -- 172.21.15.12:0/2469211915 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08011c20 con 0x7f8f181484e0 2024-04-27T07:04:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f6516422640 1 -- 172.21.15.12:0/1373321917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6510099ab0 con 0x7f65100982a0 2024-04-27T07:04:26.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f650cff9640 1 -- 172.21.15.12:0/1373321917 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f650001b070 con 0x7f65100982a0 2024-04-27T07:04:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f1c821640 1 -- 172.21.15.12:0/2469211915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f18092e20 con 0x7f8f181484e0 2024-04-27T07:04:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f34197fa640 1 -- 172.21.15.12:0/270295322 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f340c01a070 con 0x7f341c0b9ac0 2024-04-27T07:04:26.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f8f1c821640 1 -- 172.21.15.12:0/2469211915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f18090940 con 0x7f8f181484e0 2024-04-27T07:04:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f650cff9640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64fc0683d0 0x7f64fc06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.041+0000 7f650cff9640 1 -- 172.21.15.12:0/1373321917 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6500085e80 con 0x7f65100982a0 2024-04-27T07:04:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f650ffff640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64fc0683d0 0x7f64fc06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f650ffff640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64fc0683d0 0x7f64fc06a890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65080af000 tx=0x7f65080a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f8f1c821640 1 -- 172.21.15.12:0/2469211915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f18090730 con 0x7f8f181484e0 2024-04-27T07:04:26.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6452925640 1 Processor -- start 2024-04-27T07:04:26.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6452925640 1 -- start start 2024-04-27T07:04:26.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6452925640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f644c0b73c0 0x7f644c1453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6452925640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 0x7f644c1458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6452925640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644c0b77a0 con 0x7f644c0b5b00 2024-04-27T07:04:26.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f34197fa640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f34080683d0 0x7f340806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6452925640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f644c0ba490 con 0x7f644c0b73c0 2024-04-27T07:04:26.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6452925640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f644c0b5ff0 con 0x7f644c0b8ec0 2024-04-27T07:04:26.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6450ec2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 0x7f644c1458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f34197fa640 1 -- 172.21.15.12:0/270295322 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f340c085de0 con 0x7f341c0b9ac0 2024-04-27T07:04:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f644bfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f644c0b73c0 0x7f644c1453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6450ec2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 0x7f644c1458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:58668/0 (socket says 172.21.15.12:58668) 2024-04-27T07:04:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6450ec2640 1 -- 172.21.15.12:0/2665860042 learned_addr learned my addr 172.21.15.12:0/2665860042 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f3420966640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f34080683d0 0x7f340806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6450ec2640 1 -- 172.21.15.12:0/2665860042 >> v2:172.21.15.53:3300/0 conn(0x7f644c0b73c0 msgr2=0x7f644c1453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:26.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6450ec2640 1 --2- 172.21.15.12:0/2665860042 >> v2:172.21.15.53:3300/0 conn(0x7f644c0b73c0 0x7f644c1453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6450ec2640 1 -- 172.21.15.12:0/2665860042 >> v1:172.21.15.53:6789/0 conn(0x7f644c0b5b00 legacy=0x7f644c0b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.045+0000 7f6450ec2640 1 -- 172.21.15.12:0/2665860042 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c0b62d0 con 0x7f644c0b8ec0 2024-04-27T07:04:26.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f3420966640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f34080683d0 0x7f340806a890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f34140af000 tx=0x7f34140a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa20fe7640 1 Processor -- start 2024-04-27T07:04:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa20fe7640 1 -- start start 2024-04-27T07:04:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f34197fa640 1 -- 172.21.15.12:0/270295322 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f340c021020 con 0x7f341c0b9ac0 2024-04-27T07:04:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa20fe7640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffa1c147420 0x7ffa1c0af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa20fe7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffa1c0b58b0 0x7ffa1c09aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa20fe7640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1c147800 con 0x7ffa1c09aa70 2024-04-27T07:04:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa20fe7640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c0af950 con 0x7ffa1c0b58b0 2024-04-27T07:04:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa20fe7640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c0afb70 con 0x7ffa1c147420 2024-04-27T07:04:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f6450ec2640 1 --2- 172.21.15.12:0/2665860042 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 0x7f644c1458f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f644400d150 tx=0x7f644401ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f4e704bd0251463 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa1ad9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffa1c0b58b0 0x7ffa1c09aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa1ad9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffa1c0b58b0 0x7ffa1c09aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42254/0 (socket says 172.21.15.12:42254) 2024-04-27T07:04:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa1ad9d640 1 -- 172.21.15.12:0/144447562 learned_addr learned my addr 172.21.15.12:0/144447562 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa1a59c640 1 --2- 172.21.15.12:0/144447562 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffa1c147420 0x7ffa1c0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f644affd640 1 -- 172.21.15.12:0/2665860042 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644400b2d0 con 0x7f644c0b8ec0 2024-04-27T07:04:26.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f644affd640 1 -- 172.21.15.12:0/2665860042 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644400b470 con 0x7f644c0b8ec0 2024-04-27T07:04:26.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f644affd640 1 -- 172.21.15.12:0/2665860042 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64440a7aa0 con 0x7f644c0b8ec0 2024-04-27T07:04:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f6452925640 1 -- 172.21.15.12:0/2665860042 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 msgr2=0x7f644c1458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f6452925640 1 --2- 172.21.15.12:0/2665860042 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 0x7f644c1458f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f644400d150 tx=0x7f644401ebc0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f6452925640 1 -- 172.21.15.12:0/2665860042 shutdown_connections 2024-04-27T07:04:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f6452925640 1 --2- 172.21.15.12:0/2665860042 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 0x7f644c1458f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f6452925640 1 --2- 172.21.15.12:0/2665860042 >> v2:172.21.15.53:3300/0 conn(0x7f644c0b73c0 0x7f644c1453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7f6452925640 1 -- 172.21.15.12:0/2665860042 >> 172.21.15.12:0/2665860042 conn(0x7f644c0ba6d0 msgr2=0x7f644c0baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa1ad9d640 1 -- 172.21.15.12:0/144447562 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffa1c147420 msgr2=0x7ffa1c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa1ad9d640 1 --2- 172.21.15.12:0/144447562 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffa1c147420 0x7ffa1c0af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa1ad9d640 1 -- 172.21.15.12:0/144447562 >> v1:172.21.15.12:6789/0 conn(0x7ffa1c09aa70 legacy=0x7ffa1c0b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.049+0000 7ffa1ad9d640 1 -- 172.21.15.12:0/144447562 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0afe50 con 0x7ffa1c0b58b0 2024-04-27T07:04:26.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7ffa1ad9d640 1 --2- 172.21.15.12:0/144447562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0b58b0 0x7ffa1c09aed0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400c110 tx=0x7ffa1401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6faef1f2ce9bb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f6452925640 1 -- 172.21.15.12:0/2665860042 shutdown_connections 2024-04-27T07:04:26.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f6452925640 1 -- 172.21.15.12:0/2665860042 wait complete. 2024-04-27T07:04:26.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f98e868c640 1 Processor -- start 2024-04-27T07:04:26.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f98e868c640 1 -- start start 2024-04-27T07:04:26.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f6452925640 1 Processor -- start 2024-04-27T07:04:26.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f6452925640 1 -- start start 2024-04-27T07:04:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f98e868c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f98e00b9d50 0x7f98e00ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f98e868c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f98e00b7e30 0x7f98e0147f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f98e5c27640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f98e00b7e30 0x7f98e0147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f6452925640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f644c0b5b00 0x7f644c13dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f6452925640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644c0b73c0 0x7f644c141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f6452925640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 0x7f644c141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f6452925640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f644c06c760 con 0x7f644c0b73c0 2024-04-27T07:04:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f6452925640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f644c06c5e0 con 0x7f644c0b5b00 2024-04-27T07:04:26.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f6452925640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f644c06c8e0 con 0x7f644c0b8ec0 2024-04-27T07:04:26.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.053+0000 7f98e5c27640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f98e00b7e30 0x7f98e0147f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42256/0 (socket says 172.21.15.12:42256) 2024-04-27T07:04:26.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f98e5c27640 1 -- 172.21.15.12:0/2071672779 learned_addr learned my addr 172.21.15.12:0/2071672779 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f98e6428640 1 --2- 172.21.15.12:0/2071672779 >> v2:172.21.15.53:3300/0 conn(0x7f98e00b9d50 0x7f98e00ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f98e868c640 1 -- 172.21.15.12:0/2071672779 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98e00b9360 con 0x7f98e01484d0 2024-04-27T07:04:26.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f98e868c640 1 -- 172.21.15.12:0/2071672779 --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f98e014bf30 con 0x7f98e00b7e30 2024-04-27T07:04:26.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f98e868c640 1 -- 172.21.15.12:0/2071672779 --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f98e014c3c0 con 0x7f98e00b9d50 2024-04-27T07:04:26.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f644b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644c0b73c0 0x7f644c141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f644b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644c0b73c0 0x7f644c141670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42258/0 (socket says 172.21.15.12:42258) 2024-04-27T07:04:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f644b7fe640 1 -- 172.21.15.12:0/3640970155 learned_addr learned my addr 172.21.15.12:0/3640970155 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f644bfff640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f644c0b5b00 0x7f644c13dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f6450ec2640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 0x7f644c141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f644b7fe640 1 -- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 msgr2=0x7f644c141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f644b7fe640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 0x7f644c141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f644b7fe640 1 -- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f644c0b5b00 msgr2=0x7f644c13dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f644b7fe640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f644c0b5b00 0x7f644c13dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f8eff7fe640 1 -- 172.21.15.12:0/2469211915 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8f08018020 con 0x7f8f181484e0 2024-04-27T07:04:26.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f644b7fe640 1 -- 172.21.15.12:0/3640970155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c1409b0 con 0x7f644c0b73c0 2024-04-27T07:04:26.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f644b7fe640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644c0b73c0 0x7f644c141670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f643c00c3f0 tx=0x7f643c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f6448ff9640 1 -- 172.21.15.12:0/3640970155 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c017070 con 0x7f644c0b73c0 2024-04-27T07:04:26.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f6452925640 1 -- 172.21.15.12:0/3640970155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644c0afe30 con 0x7f644c0b73c0 2024-04-27T07:04:26.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f6452925640 1 -- 172.21.15.12:0/3640970155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644c0b1450 con 0x7f644c0b73c0 2024-04-27T07:04:26.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f98e6428640 1 -- 172.21.15.12:0/2071672779 >> v2:172.21.15.12:3300/0 conn(0x7f98e00b7e30 msgr2=0x7f98e0147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:26.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f98e6428640 1 --2- 172.21.15.12:0/2071672779 >> v2:172.21.15.12:3300/0 conn(0x7f98e00b7e30 0x7f98e0147f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f98e6428640 1 -- 172.21.15.12:0/2071672779 >> v1:172.21.15.53:6789/0 conn(0x7f98e01484d0 legacy=0x7f98e00b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:26.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f98e6428640 1 -- 172.21.15.12:0/2071672779 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e014b8e0 con 0x7f98e00b9d50 2024-04-27T07:04:26.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f650cff9640 1 -- 172.21.15.12:0/1373321917 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6510099ab0 con 0x7f65100982a0 2024-04-27T07:04:26.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f6448ff9640 1 -- 172.21.15.12:0/3640970155 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643c0052d0 con 0x7f644c0b73c0 2024-04-27T07:04:26.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f6448ff9640 1 -- 172.21.15.12:0/3640970155 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c0069e0 con 0x7f644c0b73c0 2024-04-27T07:04:26.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f8eff7fe640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8f040683d0 0x7f8f0406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f98e6428640 1 --2- 172.21.15.12:0/2071672779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e00b9d50 0x7f98e00ba130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f98e0071b70 tx=0x7f98d001c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3568ee5ca24ef652 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7f8eff7fe640 1 -- 172.21.15.12:0/2469211915 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8f08086bb0 con 0x7f8f181484e0 2024-04-27T07:04:26.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f8f16d9d640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8f040683d0 0x7f8f0406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7ffa1959a640 1 -- 172.21.15.12:0/144447562 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa140ab070 con 0x7ffa1c0b58b0 2024-04-27T07:04:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7ffa1959a640 1 -- 172.21.15.12:0/144447562 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa140a5b30 con 0x7ffa1c0b58b0 2024-04-27T07:04:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.057+0000 7ffa1959a640 1 -- 172.21.15.12:0/144447562 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1400be60 con 0x7ffa1c0b58b0 2024-04-27T07:04:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7ffa20fe7640 1 -- 172.21.15.12:0/144447562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0b58b0 msgr2=0x7ffa1c09aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7ffa20fe7640 1 --2- 172.21.15.12:0/144447562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0b58b0 0x7ffa1c09aed0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400c110 tx=0x7ffa1401cc60 comp rx=0 tx=0).stop 2024-04-27T07:04:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e5426640 1 -- 172.21.15.12:0/2071672779 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d00b0070 con 0x7f98e00b9d50 2024-04-27T07:04:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e5426640 1 -- 172.21.15.12:0/2071672779 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d00ab600 con 0x7f98e00b9d50 2024-04-27T07:04:26.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e5426640 1 -- 172.21.15.12:0/2071672779 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d00ab9b0 con 0x7f98e00b9d50 2024-04-27T07:04:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7ffa20fe7640 1 -- 172.21.15.12:0/144447562 shutdown_connections 2024-04-27T07:04:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7ffa20fe7640 1 --2- 172.21.15.12:0/144447562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c0b58b0 0x7ffa1c09aed0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7ffa20fe7640 1 --2- 172.21.15.12:0/144447562 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffa1c147420 0x7ffa1c0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7ffa20fe7640 1 -- 172.21.15.12:0/144447562 >> 172.21.15.12:0/144447562 conn(0x7ffa1c0b5e40 msgr2=0x7ffa1c0b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7ffa20fe7640 1 -- 172.21.15.12:0/144447562 shutdown_connections 2024-04-27T07:04:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7ffa20fe7640 1 -- 172.21.15.12:0/144447562 wait complete. 2024-04-27T07:04:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e868c640 1 -- 172.21.15.12:0/2071672779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e00b9d50 msgr2=0x7f98e00ba130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e868c640 1 --2- 172.21.15.12:0/2071672779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e00b9d50 0x7f98e00ba130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f98e0071b70 tx=0x7f98d001c340 comp rx=0 tx=0).stop 2024-04-27T07:04:26.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7ffa20fe7640 1 Processor -- start 2024-04-27T07:04:26.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e868c640 1 -- 172.21.15.12:0/2071672779 shutdown_connections 2024-04-27T07:04:26.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e868c640 1 --2- 172.21.15.12:0/2071672779 >> v2:172.21.15.12:3300/0 conn(0x7f98e00b7e30 0x7f98e0147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e868c640 1 --2- 172.21.15.12:0/2071672779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e00b9d50 0x7f98e00ba130 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e868c640 1 -- 172.21.15.12:0/2071672779 >> 172.21.15.12:0/2071672779 conn(0x7f98e00ba6d0 msgr2=0x7f98e00baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e868c640 1 -- 172.21.15.12:0/2071672779 shutdown_connections 2024-04-27T07:04:26.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7f98e868c640 1 -- 172.21.15.12:0/2071672779 wait complete. 2024-04-27T07:04:26.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.061+0000 7ffa20fe7640 1 -- start start 2024-04-27T07:04:26.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa20fe7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c09aa70 0x7ffa1c0b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa20fe7640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffa1c0b58b0 0x7ffa1c141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa20fe7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa1c147420 0x7ffa1c13c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa20fe7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c150f20 con 0x7ffa1c09aa70 2024-04-27T07:04:26.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa20fe7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c1510a0 con 0x7ffa1c147420 2024-04-27T07:04:26.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa20fe7640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c151220 con 0x7ffa1c0b58b0 2024-04-27T07:04:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa1a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c09aa70 0x7ffa1c0b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa1a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c09aa70 0x7ffa1c0b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42266/0 (socket says 172.21.15.12:42266) 2024-04-27T07:04:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa1a59c640 1 -- 172.21.15.12:0/1867648744 learned_addr learned my addr 172.21.15.12:0/1867648744 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa19d9b640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffa1c0b58b0 0x7ffa1c141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa1ad9d640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa1c147420 0x7ffa1c13c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e868c640 1 Processor -- start 2024-04-27T07:04:26.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e868c640 1 -- start start 2024-04-27T07:04:26.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa1a59c640 1 -- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffa1c0b58b0 msgr2=0x7ffa1c141ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa1a59c640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffa1c0b58b0 0x7ffa1c141ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa1a59c640 1 -- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa1c147420 msgr2=0x7ffa1c13c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa1a59c640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa1c147420 0x7ffa1c13c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa1a59c640 1 -- 172.21.15.12:0/1867648744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c142000 con 0x7ffa1c09aa70 2024-04-27T07:04:26.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e868c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98e00b7e30 0x7f98e009e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7ffa1a59c640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c09aa70 0x7ffa1c0b13c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00d860 tx=0x7ffa0c00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e868c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00b9d50 0x7f98e01e8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e868c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01484d0 0x7f98e01e8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e868c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f98e01551d0 con 0x7f98e00b9d50 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e868c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0155050 con 0x7f98e01484d0 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e868c640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f98e0155350 con 0x7f98e00b7e30 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e5c27640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00b9d50 0x7f98e01e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e5c27640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00b9d50 0x7f98e01e8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42278/0 (socket says 172.21.15.12:42278) 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e5c27640 1 -- 172.21.15.12:0/172873205 learned_addr learned my addr 172.21.15.12:0/172873205 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e6428640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98e00b7e30 0x7f98e009e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e6c29640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01484d0 0x7f98e01e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e5c27640 1 -- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98e00b7e30 msgr2=0x7f98e009e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e5c27640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98e00b7e30 0x7f98e009e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e5c27640 1 -- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01484d0 msgr2=0x7f98e01e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e5c27640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01484d0 0x7f98e01e8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e5c27640 1 -- 172.21.15.12:0/172873205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e008fee0 con 0x7f98e00b9d50 2024-04-27T07:04:26.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.065+0000 7f98e5c27640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00b9d50 0x7f98e01e8240 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00e7f0 tx=0x7f98dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.069+0000 7f8f16d9d640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8f040683d0 0x7f8f0406a890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f10005c10 tx=0x7f8f10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.069+0000 7f8eff7fe640 1 -- 172.21.15.12:0/2469211915 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8f0801f020 con 0x7f8f181484e0 2024-04-27T07:04:26.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.069+0000 7f6448ff9640 1 -- 172.21.15.12:0/3640970155 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f643c01a070 con 0x7f644c0b73c0 2024-04-27T07:04:26.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.069+0000 7f6448ff9640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6438068380 0x7f643806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.073+0000 7f644bfff640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6438068380 0x7f643806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.073+0000 7f6448ff9640 1 -- 172.21.15.12:0/3640970155 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f643c053c50 con 0x7f644c0b73c0 2024-04-27T07:04:26.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.073+0000 7f644bfff640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6438068380 0x7f643806a840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6440009af0 tx=0x7f6440008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.073+0000 7ffa0affd640 1 -- 172.21.15.12:0/1867648744 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c01a070 con 0x7ffa1c09aa70 2024-04-27T07:04:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.073+0000 7ffa0affd640 1 -- 172.21.15.12:0/1867648744 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0c00de50 con 0x7ffa1c09aa70 2024-04-27T07:04:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.073+0000 7ffa20fe7640 1 -- 172.21.15.12:0/1867648744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1c0b1aa0 con 0x7ffa1c09aa70 2024-04-27T07:04:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.073+0000 7ffa0affd640 1 -- 172.21.15.12:0/1867648744 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c00cb60 con 0x7ffa1c09aa70 2024-04-27T07:04:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.073+0000 7ffa20fe7640 1 -- 172.21.15.12:0/1867648744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1c1e4b30 con 0x7ffa1c09aa70 2024-04-27T07:04:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.073+0000 7ffa20fe7640 1 -- 172.21.15.12:0/1867648744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa1c068c20 con 0x7ffa1c09aa70 2024-04-27T07:04:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7ffa0affd640 1 -- 172.21.15.12:0/1867648744 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffa0c01d070 con 0x7ffa1c09aa70 2024-04-27T07:04:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7f6452925640 1 -- 172.21.15.12:0/3640970155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642c0052a0 con 0x7f644c0b73c0 2024-04-27T07:04:26.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7ffa0affd640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff9fc068480 0x7ff9fc06a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7f98d6ffd640 1 -- 172.21.15.12:0/172873205 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc017070 con 0x7f98e00b9d50 2024-04-27T07:04:26.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7ffa0affd640 1 -- 172.21.15.12:0/1867648744 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffa0c085e80 con 0x7ffa1c09aa70 2024-04-27T07:04:26.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7ffa19d9b640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff9fc068480 0x7ff9fc06a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7f98e868c640 1 -- 172.21.15.12:0/172873205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e00b6950 con 0x7f98e00b9d50 2024-04-27T07:04:26.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7f98e868c640 1 -- 172.21.15.12:0/172873205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e009f450 con 0x7f98e00b9d50 2024-04-27T07:04:26.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7f98d6ffd640 1 -- 172.21.15.12:0/172873205 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98dc00c940 con 0x7f98e00b9d50 2024-04-27T07:04:26.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7f98d6ffd640 1 -- 172.21.15.12:0/172873205 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc00cd30 con 0x7f98e00b9d50 2024-04-27T07:04:26.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.077+0000 7ffa19d9b640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff9fc068480 0x7ff9fc06a940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c1369e0 tx=0x7ffa10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.081+0000 7f98e868c640 1 -- 172.21.15.12:0/172873205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c80052a0 con 0x7f98e00b9d50 2024-04-27T07:04:26.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.085+0000 7ffa0affd640 1 -- 172.21.15.12:0/1867648744 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffa0c055e60 con 0x7ffa1c09aa70 2024-04-27T07:04:26.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.093+0000 7f38fbfff640 1 Processor -- start 2024-04-27T07:04:26.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.093+0000 7f38fbfff640 1 -- start start 2024-04-27T07:04:26.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.093+0000 7f6448ff9640 1 -- 172.21.15.12:0/3640970155 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f643c005480 con 0x7f644c0b73c0 2024-04-27T07:04:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.093+0000 7f38fbfff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f38f4037c20 0x7f38f40c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38fbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f38f40c33d0 0x7f38f40c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38fbfff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38f40c9c40 0x7f38f4036c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38fbfff640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f38f40367d0 con 0x7f38f40c33d0 2024-04-27T07:04:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38fbfff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f38f402aca0 con 0x7f38f4037c20 2024-04-27T07:04:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38fbfff640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f38f4038000 con 0x7f38f40c9c40 2024-04-27T07:04:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38fb7fe640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38f40c9c40 0x7f38f4036c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38faffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f38f4037c20 0x7f38f40c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38faffd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f38f4037c20 0x7f38f40c2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:37282/0 (socket says 172.21.15.12:37282) 2024-04-27T07:04:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38faffd640 1 -- 172.21.15.12:0/1814439369 learned_addr learned my addr 172.21.15.12:0/1814439369 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38faffd640 1 -- 172.21.15.12:0/1814439369 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38f40c9c40 msgr2=0x7f38f4036c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:26.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38faffd640 1 --2- 172.21.15.12:0/1814439369 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38f40c9c40 0x7f38f4036c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38faffd640 1 -- 172.21.15.12:0/1814439369 >> v2:172.21.15.12:3300/0 conn(0x7f38f40c33d0 msgr2=0x7f38f40c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:26.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38faffd640 1 --2- 172.21.15.12:0/1814439369 >> v2:172.21.15.12:3300/0 conn(0x7f38f40c33d0 0x7f38f40c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.097+0000 7f38faffd640 1 -- 172.21.15.12:0/1814439369 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f4036460 con 0x7f38f4037c20 2024-04-27T07:04:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38faffd640 1 --2- 172.21.15.12:0/1814439369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f4037c20 0x7f38f40c2e90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38f403f990 tx=0x7f38ec020820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c160cdd26f514b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38f9ffb640 1 -- 172.21.15.12:0/1814439369 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0ac610 con 0x7f38f4037c20 2024-04-27T07:04:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38f9ffb640 1 -- 172.21.15.12:0/1814439369 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ec0ac7b0 con 0x7f38f4037c20 2024-04-27T07:04:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38fbfff640 1 -- 172.21.15.12:0/1814439369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f4037c20 msgr2=0x7f38f40c2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38fbfff640 1 --2- 172.21.15.12:0/1814439369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f4037c20 0x7f38f40c2e90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38f403f990 tx=0x7f38ec020820 comp rx=0 tx=0).stop 2024-04-27T07:04:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38fbfff640 1 -- 172.21.15.12:0/1814439369 shutdown_connections 2024-04-27T07:04:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38fbfff640 1 --2- 172.21.15.12:0/1814439369 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38f40c9c40 0x7f38f4036c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38fbfff640 1 --2- 172.21.15.12:0/1814439369 >> v2:172.21.15.12:3300/0 conn(0x7f38f40c33d0 0x7f38f40c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38fbfff640 1 --2- 172.21.15.12:0/1814439369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f4037c20 0x7f38f40c2e90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38fbfff640 1 -- 172.21.15.12:0/1814439369 >> 172.21.15.12:0/1814439369 conn(0x7f38f4008710 msgr2=0x7f38f403eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38fbfff640 1 -- 172.21.15.12:0/1814439369 shutdown_connections 2024-04-27T07:04:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f98d6ffd640 1 -- 172.21.15.12:0/172873205 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f98dc01a070 con 0x7f98e00b9d50 2024-04-27T07:04:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f98d6ffd640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f98c40683d0 0x7f98c406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f98d6ffd640 1 -- 172.21.15.12:0/172873205 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f98dc086e30 con 0x7f98e00b9d50 2024-04-27T07:04:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f98e6428640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f98c40683d0 0x7f98c406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f98e6428640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f98c40683d0 0x7f98c406a890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f98d00ad000 tx=0x7f98d00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.101+0000 7f38fbfff640 1 -- 172.21.15.12:0/1814439369 wait complete. 2024-04-27T07:04:26.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fbfff640 1 Processor -- start 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fbfff640 1 -- start start 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f40c33d0 0x7f38f40d3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f40c9c40 0x7f38f4171cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fbfff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38f40d22c0 0x7f38f4172200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fbfff640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f38f40cd1f0 con 0x7f38f40c9c40 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fbfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f38f40ca050 con 0x7f38f40c33d0 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fbfff640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f38f40cd370 con 0x7f38f40d22c0 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fa7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f40c9c40 0x7f38f4171cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fa7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f40c9c40 0x7f38f4171cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42302/0 (socket says 172.21.15.12:42302) 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fa7fc640 1 -- 172.21.15.12:0/2767275805 learned_addr learned my addr 172.21.15.12:0/2767275805 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fb7fe640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38f40d22c0 0x7f38f4172200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38faffd640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f40c33d0 0x7f38f40d3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fa7fc640 1 -- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38f40d22c0 msgr2=0x7f38f4172200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fa7fc640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38f40d22c0 0x7f38f4172200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fa7fc640 1 -- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f40c33d0 msgr2=0x7f38f40d3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fa7fc640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f40c33d0 0x7f38f40d3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fa7fc640 1 -- 172.21.15.12:0/2767275805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f40d27f0 con 0x7f38f40c9c40 2024-04-27T07:04:26.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.105+0000 7f38fa7fc640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f40c9c40 0x7f38f4171cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38f000b550 tx=0x7f38f000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.109+0000 7f98d6ffd640 1 -- 172.21.15.12:0/172873205 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f98dc056f10 con 0x7f98e00b9d50 2024-04-27T07:04:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.113+0000 7f38e37fe640 1 -- 172.21.15.12:0/2767275805 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0014070 con 0x7f38f40c9c40 2024-04-27T07:04:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.113+0000 7f38fbfff640 1 -- 172.21.15.12:0/2767275805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f40d00c0 con 0x7f38f40c9c40 2024-04-27T07:04:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.113+0000 7f38fbfff640 1 -- 172.21.15.12:0/2767275805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f40cfc40 con 0x7f38f40c9c40 2024-04-27T07:04:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.113+0000 7f38e37fe640 1 -- 172.21.15.12:0/2767275805 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f0010890 con 0x7f38f40c9c40 2024-04-27T07:04:26.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.113+0000 7f38e37fe640 1 -- 172.21.15.12:0/2767275805 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0010c80 con 0x7f38f40c9c40 2024-04-27T07:04:26.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.113+0000 7f38fbfff640 1 -- 172.21.15.12:0/2767275805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc0052a0 con 0x7f38f40c9c40 2024-04-27T07:04:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.113+0000 7f38e37fe640 1 -- 172.21.15.12:0/2767275805 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f38f0017070 con 0x7f38f40c9c40 2024-04-27T07:04:26.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.113+0000 7f38e37fe640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38e8068450 0x7f38e806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:26.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.113+0000 7f38e37fe640 1 -- 172.21.15.12:0/2767275805 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f38f0086690 con 0x7f38f40c9c40 2024-04-27T07:04:26.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.117+0000 7f38faffd640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38e8068450 0x7f38e806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:26.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.117+0000 7f38faffd640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38e8068450 0x7f38e806a910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0b3000 tx=0x7f38ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:26.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.125+0000 7f38e37fe640 1 -- 172.21.15.12:0/2767275805 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f38f008b080 con 0x7f38f40c9c40 2024-04-27T07:04:26.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.369+0000 7f704748f640 1 -- 172.21.15.12:0/3289521975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f704009b170 con 0x7f70400b51a0 2024-04-27T07:04:26.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.373+0000 7f7035ffb640 1 -- 172.21.15.12:0/3289521975 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7038056c30 con 0x7f70400b51a0 2024-04-27T07:04:26.379 INFO:teuthology.orchestra.run.smithi012.stdout:55834574853 2024-04-27T07:04:26.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.373+0000 7f702b7fe640 1 -- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70240683d0 msgr2=0x7f702406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.373+0000 7f702b7fe640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70240683d0 0x7f702406a890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f70300b1000 tx=0x7f7030009040 comp rx=0 tx=0).stop 2024-04-27T07:04:26.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.377+0000 7f702b7fe640 1 -- 172.21.15.12:0/3289521975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70400b51a0 msgr2=0x7f7040136800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.377+0000 7f702b7fe640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70400b51a0 0x7f7040136800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f703800e3b0 tx=0x7f703800e870 comp rx=0 tx=0).stop 2024-04-27T07:04:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.377+0000 7f702b7fe640 1 -- 172.21.15.12:0/3289521975 shutdown_connections 2024-04-27T07:04:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.377+0000 7f702b7fe640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70240683d0 0x7f702406a890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.377+0000 7f702b7fe640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7040147420 0x7f7040141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.377+0000 7f702b7fe640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70400b51a0 0x7f7040136800 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.377+0000 7f702b7fe640 1 --2- 172.21.15.12:0/3289521975 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f704009aa70 0x7f704013c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.377+0000 7f702b7fe640 1 -- 172.21.15.12:0/3289521975 >> 172.21.15.12:0/3289521975 conn(0x7f70400b5e40 msgr2=0x7f70401392a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.377+0000 7f702b7fe640 1 -- 172.21.15.12:0/3289521975 shutdown_connections 2024-04-27T07:04:26.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.377+0000 7f702b7fe640 1 -- 172.21.15.12:0/3289521975 wait complete. 2024-04-27T07:04:26.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.381+0000 7f6516422640 1 -- 172.21.15.12:0/1373321917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6510069330 con 0x7f65100982a0 2024-04-27T07:04:26.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.381+0000 7f650cff9640 1 -- 172.21.15.12:0/1373321917 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6500056520 con 0x7f65100982a0 2024-04-27T07:04:26.386 INFO:teuthology.orchestra.run.smithi012.stdout:55834574853 2024-04-27T07:04:26.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.381+0000 7f3422bca640 1 -- 172.21.15.12:0/270295322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f341c0b55b0 con 0x7f341c0b9ac0 2024-04-27T07:04:26.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f34197fa640 1 -- 172.21.15.12:0/270295322 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f340c055f80 con 0x7f341c0b9ac0 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 -- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64fc0683d0 msgr2=0x7f64fc06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64fc0683d0 0x7f64fc06a890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65080af000 tx=0x7f65080a7070 comp rx=0 tx=0).stop 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 -- 172.21.15.12:0/1373321917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100982a0 msgr2=0x7f6510098710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100982a0 0x7f6510098710 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f650000c780 tx=0x7f650000cc40 comp rx=0 tx=0).stop 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 -- 172.21.15.12:0/1373321917 shutdown_connections 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64fc0683d0 0x7f64fc06a890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65100982a0 0x7f6510098710 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f65100adee0 0x7f6510099570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 --2- 172.21.15.12:0/1373321917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f651006bb80 0x7f65100b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 -- 172.21.15.12:0/1373321917 >> 172.21.15.12:0/1373321917 conn(0x7f6510067f80 msgr2=0x7f65100b8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 -- 172.21.15.12:0/1373321917 shutdown_connections 2024-04-27T07:04:26.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f8f1c821640 1 -- 172.21.15.12:0/2469211915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8f180b97c0 con 0x7f8f181484e0 2024-04-27T07:04:26.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f6516422640 1 -- 172.21.15.12:0/1373321917 wait complete. 2024-04-27T07:04:26.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.385+0000 7f8eff7fe640 1 -- 172.21.15.12:0/2469211915 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f08056c90 con 0x7f8f181484e0 2024-04-27T07:04:26.391 INFO:teuthology.orchestra.run.smithi012.stdout:55834574855 2024-04-27T07:04:26.392 INFO:teuthology.orchestra.run.smithi012.stdout:55834574853 2024-04-27T07:04:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 -- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8f040683d0 msgr2=0x7f8f0406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8f040683d0 0x7f8f0406a890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f10005c10 tx=0x7f8f10009040 comp rx=0 tx=0).stop 2024-04-27T07:04:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 -- 172.21.15.12:0/2469211915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f181484e0 msgr2=0x7f8f181e1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f181484e0 0x7f8f181e1000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800b3e0 tx=0x7f8f0800b8a0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 -- 172.21.15.12:0/2469211915 shutdown_connections 2024-04-27T07:04:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8f040683d0 0x7f8f0406a890 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f181484e0 0x7f8f181e1000 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f180b82c0 0x7f8f181e0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 --2- 172.21.15.12:0/2469211915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8f1806bde0 0x7f8f18091110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 -- 172.21.15.12:0/2469211915 >> 172.21.15.12:0/2469211915 conn(0x7f8f1809c270 msgr2=0x7f8f1806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 -- 172.21.15.12:0/2469211915 shutdown_connections 2024-04-27T07:04:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f8efd7fa640 1 -- 172.21.15.12:0/2469211915 wait complete. 2024-04-27T07:04:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f3422bca640 1 -- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f34080683d0 msgr2=0x7f340806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f3422bca640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f34080683d0 0x7f340806a890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f34140af000 tx=0x7f34140a7070 comp rx=0 tx=0).stop 2024-04-27T07:04:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f3422bca640 1 -- 172.21.15.12:0/270295322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f341c0b9ac0 msgr2=0x7f341c143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.389+0000 7f3422bca640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f341c0b9ac0 0x7f341c143d30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c520 tx=0x7f340c00c9e0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.393+0000 7f3422bca640 1 -- 172.21.15.12:0/270295322 shutdown_connections 2024-04-27T07:04:26.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.393+0000 7f3422bca640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f34080683d0 0x7f340806a890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.393+0000 7f3422bca640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f341c0b9ac0 0x7f341c143d30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.393+0000 7f3422bca640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f341c0b4720 0x7f341c072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.393+0000 7f3422bca640 1 --2- 172.21.15.12:0/270295322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f341c06e3c0 0x7f341c1ed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.393+0000 7f3422bca640 1 -- 172.21.15.12:0/270295322 >> 172.21.15.12:0/270295322 conn(0x7f341c09aa70 msgr2=0x7f341c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.393+0000 7f3422bca640 1 -- 172.21.15.12:0/270295322 shutdown_connections 2024-04-27T07:04:26.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.393+0000 7f3422bca640 1 -- 172.21.15.12:0/270295322 wait complete. 2024-04-27T07:04:26.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-04-27T07:04:26.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-04-27T07:04:26.421 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-04-27T07:04:26.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.417+0000 7ffa20fe7640 1 -- 172.21.15.12:0/1867648744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ffa1c0af360 con 0x7ffa1c09aa70 2024-04-27T07:04:26.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.417+0000 7ffa0affd640 1 -- 172.21.15.12:0/1867648744 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffa0c05a160 con 0x7ffa1c09aa70 2024-04-27T07:04:26.422 INFO:teuthology.orchestra.run.smithi012.stdout:55834574853 2024-04-27T07:04:26.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 -- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff9fc068480 msgr2=0x7ff9fc06a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff9fc068480 0x7ff9fc06a940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c1369e0 tx=0x7ffa10009040 comp rx=0 tx=0).stop 2024-04-27T07:04:26.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 -- 172.21.15.12:0/1867648744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c09aa70 msgr2=0x7ffa1c0b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c09aa70 0x7ffa1c0b13c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00d860 tx=0x7ffa0c00dd20 comp rx=0 tx=0).stop 2024-04-27T07:04:26.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 -- 172.21.15.12:0/1867648744 shutdown_connections 2024-04-27T07:04:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff9fc068480 0x7ff9fc06a940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa1c147420 0x7ffa1c13c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ffa1c0b58b0 0x7ffa1c141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 --2- 172.21.15.12:0/1867648744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa1c09aa70 0x7ffa1c0b13c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 -- 172.21.15.12:0/1867648744 >> 172.21.15.12:0/1867648744 conn(0x7ffa1c0b5e40 msgr2=0x7ffa1c0b7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 -- 172.21.15.12:0/1867648744 shutdown_connections 2024-04-27T07:04:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.421+0000 7ffa20fe7640 1 -- 172.21.15.12:0/1867648744 wait complete. 2024-04-27T07:04:26.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2024-04-27T07:04:26.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.425+0000 7f6452925640 1 -- 172.21.15.12:0/3640970155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f642c005860 con 0x7f644c0b73c0 2024-04-27T07:04:26.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.425+0000 7f6448ff9640 1 -- 172.21.15.12:0/3640970155 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f643c0562b0 con 0x7f644c0b73c0 2024-04-27T07:04:26.432 INFO:teuthology.orchestra.run.smithi012.stdout:55834574853 2024-04-27T07:04:26.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f98e868c640 1 -- 172.21.15.12:0/172873205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f98c8005860 con 0x7f98e00b9d50 2024-04-27T07:04:26.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f98d6ffd640 1 -- 172.21.15.12:0/172873205 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f98dc05b210 con 0x7f98e00b9d50 2024-04-27T07:04:26.435 INFO:teuthology.orchestra.run.smithi012.stdout:55834574852 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 -- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6438068380 msgr2=0x7f643806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6438068380 0x7f643806a840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6440009af0 tx=0x7f6440008040 comp rx=0 tx=0).stop 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 -- 172.21.15.12:0/3640970155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644c0b73c0 msgr2=0x7f644c141670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644c0b73c0 0x7f644c141670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f643c00c3f0 tx=0x7f643c00c8b0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 -- 172.21.15.12:0/3640970155 shutdown_connections 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6438068380 0x7f643806a840 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f644c0b8ec0 0x7f644c141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644c0b73c0 0x7f644c141670 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 --2- 172.21.15.12:0/3640970155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f644c0b5b00 0x7f644c13dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 -- 172.21.15.12:0/3640970155 >> 172.21.15.12:0/3640970155 conn(0x7f644c0ba6d0 msgr2=0x7f644c145180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 -- 172.21.15.12:0/3640970155 shutdown_connections 2024-04-27T07:04:26.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.429+0000 7f6452925640 1 -- 172.21.15.12:0/3640970155 wait complete. 2024-04-27T07:04:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 -- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f98c40683d0 msgr2=0x7f98c406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f98c40683d0 0x7f98c406a890 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f98d00ad000 tx=0x7f98d00a6000 comp rx=0 tx=0).stop 2024-04-27T07:04:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 -- 172.21.15.12:0/172873205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00b9d50 msgr2=0x7f98e01e8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00b9d50 0x7f98e01e8240 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00e7f0 tx=0x7f98dc00ecb0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 -- 172.21.15.12:0/172873205 shutdown_connections 2024-04-27T07:04:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f98c40683d0 0x7f98c406a890 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98e01484d0 0x7f98e01e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e00b9d50 0x7f98e01e8240 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 --2- 172.21.15.12:0/172873205 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f98e00b7e30 0x7f98e009e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 -- 172.21.15.12:0/172873205 >> 172.21.15.12:0/172873205 conn(0x7f98e00ba6d0 msgr2=0x7f98e006e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 -- 172.21.15.12:0/172873205 shutdown_connections 2024-04-27T07:04:26.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.433+0000 7f98e868c640 1 -- 172.21.15.12:0/172873205 wait complete. 2024-04-27T07:04:26.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-04-27T07:04:26.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-04-27T07:04:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.449+0000 7f38fbfff640 1 -- 172.21.15.12:0/2767275805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f38dc005590 con 0x7f38f40c9c40 2024-04-27T07:04:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.449+0000 7f38e37fe640 1 -- 172.21.15.12:0/2767275805 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f38f0010e20 con 0x7f38f40c9c40 2024-04-27T07:04:26.456 INFO:teuthology.orchestra.run.smithi012.stdout:55834574853 2024-04-27T07:04:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 -- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38e8068450 msgr2=0x7f38e806a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38e8068450 0x7f38e806a910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0b3000 tx=0x7f38ec008040 comp rx=0 tx=0).stop 2024-04-27T07:04:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 -- 172.21.15.12:0/2767275805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f40c9c40 msgr2=0x7f38f4171cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f40c9c40 0x7f38f4171cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38f000b550 tx=0x7f38f000ba10 comp rx=0 tx=0).stop 2024-04-27T07:04:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 -- 172.21.15.12:0/2767275805 shutdown_connections 2024-04-27T07:04:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38e8068450 0x7f38e806a910 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38f40d22c0 0x7f38f4172200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f40c9c40 0x7f38f4171cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 --2- 172.21.15.12:0/2767275805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38f40c33d0 0x7f38f40d3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 -- 172.21.15.12:0/2767275805 >> 172.21.15.12:0/2767275805 conn(0x7f38f4008710 msgr2=0x7f38f4006b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 -- 172.21.15.12:0/2767275805 shutdown_connections 2024-04-27T07:04:26.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:26.453+0000 7f38fbfff640 1 -- 172.21.15.12:0/2767275805 wait complete. 2024-04-27T07:04:26.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-04-27T07:04:26.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-04-27T07:04:27.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T07:04:27.421 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-27T07:04:27.421 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T07:04:27.431 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-27T07:04:27.446 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-27T07:04:27.456 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T07:04:27.462 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-27T07:04:27.475 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T07:04:27.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22fb91e640 1 Processor -- start 2024-04-27T07:04:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22fb91e640 1 -- start start 2024-04-27T07:04:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22fb91e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f4147420 0x7f22f40af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22fb91e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f22f40b58b0 0x7f22f409aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22fb91e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f4147800 con 0x7f22f409aa70 2024-04-27T07:04:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22fb91e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f22f40af950 con 0x7f22f40b58b0 2024-04-27T07:04:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22fb91e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f22f40afb70 con 0x7f22f4147420 2024-04-27T07:04:27.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22f9ebb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f22f40b58b0 0x7f22f409aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22f9ebb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f22f40b58b0 0x7f22f409aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55502/0 (socket says 172.21.15.12:55502) 2024-04-27T07:04:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22f9ebb640 1 -- 172.21.15.12:0/1529677262 learned_addr learned my addr 172.21.15.12:0/1529677262 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22f9ebb640 1 -- 172.21.15.12:0/1529677262 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f4147420 msgr2=0x7f22f40af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:27.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.581+0000 7f22f96ba640 1 --2- 172.21.15.12:0/1529677262 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f4147420 0x7f22f40af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.585+0000 7f22f9ebb640 1 --2- 172.21.15.12:0/1529677262 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f4147420 0x7f22f40af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.585+0000 7f22f9ebb640 1 -- 172.21.15.12:0/1529677262 >> v1:172.21.15.53:6789/0 conn(0x7f22f409aa70 legacy=0x7f22f40b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.585+0000 7f22f9ebb640 1 -- 172.21.15.12:0/1529677262 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f40afe50 con 0x7f22f40b58b0 2024-04-27T07:04:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.585+0000 7f22f96ba640 1 --2- 172.21.15.12:0/1529677262 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f4147420 0x7f22f40af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:27.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.585+0000 7f22f9ebb640 1 --2- 172.21.15.12:0/1529677262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 0x7f22f409aed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d370 tx=0x7f22ec01ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a5409905d5b061 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22ebfff640 1 -- 172.21.15.12:0/1529677262 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0ad070 con 0x7f22f40b58b0 2024-04-27T07:04:27.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22ebfff640 1 -- 172.21.15.12:0/1529677262 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ec0a7c20 con 0x7f22f40b58b0 2024-04-27T07:04:27.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22fb91e640 1 -- 172.21.15.12:0/1529677262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 msgr2=0x7f22f409aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22fb91e640 1 --2- 172.21.15.12:0/1529677262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 0x7f22f409aed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d370 tx=0x7f22ec01ed90 comp rx=0 tx=0).stop 2024-04-27T07:04:27.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22fb91e640 1 -- 172.21.15.12:0/1529677262 shutdown_connections 2024-04-27T07:04:27.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22fb91e640 1 --2- 172.21.15.12:0/1529677262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 0x7f22f409aed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22fb91e640 1 --2- 172.21.15.12:0/1529677262 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f4147420 0x7f22f40af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22fb91e640 1 -- 172.21.15.12:0/1529677262 >> 172.21.15.12:0/1529677262 conn(0x7f22f40b5e40 msgr2=0x7f22f40b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22fb91e640 1 -- 172.21.15.12:0/1529677262 shutdown_connections 2024-04-27T07:04:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22fb91e640 1 -- 172.21.15.12:0/1529677262 wait complete. 2024-04-27T07:04:27.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22fb91e640 1 Processor -- start 2024-04-27T07:04:27.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.589+0000 7f22fb91e640 1 -- start start 2024-04-27T07:04:27.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22fb91e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f409aa70 0x7f22f4073710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22fb91e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 0x7f22f41edc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22fb91e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f4147420 0x7f22f413eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22fb91e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4150f20 con 0x7f22f40b58b0 2024-04-27T07:04:27.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22fb91e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41510a0 con 0x7f22f4147420 2024-04-27T07:04:27.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22fb91e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f22f4151220 con 0x7f22f409aa70 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f8eb9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 0x7f22f41edc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f8eb9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 0x7f22f41edc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55516/0 (socket says 172.21.15.12:55516) 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f8eb9640 1 -- 172.21.15.12:0/254882703 learned_addr learned my addr 172.21.15.12:0/254882703 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f96ba640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f409aa70 0x7f22f4073710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f9ebb640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f4147420 0x7f22f413eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f8eb9640 1 -- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f409aa70 msgr2=0x7f22f4073710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f8eb9640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f409aa70 0x7f22f4073710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f8eb9640 1 -- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f4147420 msgr2=0x7f22f413eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f8eb9640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f4147420 0x7f22f413eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f8eb9640 1 -- 172.21.15.12:0/254882703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f4073db0 con 0x7f22f40b58b0 2024-04-27T07:04:27.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22f8eb9640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 0x7f22f41edc60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c450 tx=0x7f22e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22e9ffb640 1 -- 172.21.15.12:0/254882703 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4011790 con 0x7f22f40b58b0 2024-04-27T07:04:27.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22e9ffb640 1 -- 172.21.15.12:0/254882703 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e4017070 con 0x7f22f40b58b0 2024-04-27T07:04:27.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22fb91e640 1 -- 172.21.15.12:0/254882703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f413f630 con 0x7f22f40b58b0 2024-04-27T07:04:27.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22e9ffb640 1 -- 172.21.15.12:0/254882703 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4011b90 con 0x7f22f40b58b0 2024-04-27T07:04:27.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.593+0000 7f22fb91e640 1 -- 172.21.15.12:0/254882703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f4069330 con 0x7f22f40b58b0 2024-04-27T07:04:27.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.597+0000 7f22fb91e640 1 -- 172.21.15.12:0/254882703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f4068c20 con 0x7f22f40b58b0 2024-04-27T07:04:27.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.597+0000 7f22e9ffb640 1 -- 172.21.15.12:0/254882703 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f22e4011d30 con 0x7f22f40b58b0 2024-04-27T07:04:27.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.597+0000 7f22e9ffb640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22d8068380 0x7f22d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.601+0000 7f22f96ba640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22d8068380 0x7f22d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.601+0000 7f22e9ffb640 1 -- 172.21.15.12:0/254882703 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f22e4084340 con 0x7f22f40b58b0 2024-04-27T07:04:27.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.601+0000 7f22f96ba640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22d8068380 0x7f22d806a840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22f0006fd0 tx=0x7f22f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7fb89ffff640 1 Processor -- start 2024-04-27T07:04:27.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7fb89ffff640 1 -- start start 2024-04-27T07:04:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7fb89ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb8980097d0 0x7fb898008f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7f22e9ffb640 1 -- 172.21.15.12:0/254882703 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f22e4055d80 con 0x7f22f40b58b0 2024-04-27T07:04:27.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7fb89ffff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980395f0 0x7fb898039aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7fb89ffff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb898009bb0 con 0x7fb8980cc6c0 2024-04-27T07:04:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7fb89ffff640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fb8980072e0 con 0x7fb8980097d0 2024-04-27T07:04:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7fb89ffff640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fb898008600 con 0x7fb8980395f0 2024-04-27T07:04:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7fb89e7fc640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fb8980cc6c0 0x7fb8980ccb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:35506/0 (socket says 172.21.15.12:35506) 2024-04-27T07:04:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7fb89e7fc640 1 -- 172.21.15.12:0/826518378 learned_addr learned my addr 172.21.15.12:0/826518378 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:27.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.605+0000 7fb89f7fe640 1 --2- 172.21.15.12:0/826518378 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980395f0 0x7fb898039aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.609+0000 7fb89f7fe640 1 -- 172.21.15.12:0/826518378 >> v2:172.21.15.12:3300/0 conn(0x7fb8980097d0 msgr2=0x7fb898008f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.609+0000 7fb89f7fe640 1 --2- 172.21.15.12:0/826518378 >> v2:172.21.15.12:3300/0 conn(0x7fb8980097d0 0x7fb898008f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.609+0000 7fb89f7fe640 1 -- 172.21.15.12:0/826518378 >> v1:172.21.15.53:6789/0 conn(0x7fb8980cc6c0 legacy=0x7fb8980ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.609+0000 7fb89f7fe640 1 -- 172.21.15.12:0/826518378 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb898007cd0 con 0x7fb8980395f0 2024-04-27T07:04:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.609+0000 7fb89f7fe640 1 --2- 172.21.15.12:0/826518378 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980395f0 0x7fb898039aa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0050cf0 tx=0x7fb8a0097ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c0092c0c37eb99 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.609+0000 7fb89dffb640 1 -- 172.21.15.12:0/826518378 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0120b80 con 0x7fb8980395f0 2024-04-27T07:04:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89dffb640 1 -- 172.21.15.12:0/826518378 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a0120d60 con 0x7fb8980395f0 2024-04-27T07:04:27.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89dffb640 1 -- 172.21.15.12:0/826518378 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0128640 con 0x7fb8980395f0 2024-04-27T07:04:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89ffff640 1 -- 172.21.15.12:0/826518378 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980395f0 msgr2=0x7fb898039aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89ffff640 1 --2- 172.21.15.12:0/826518378 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980395f0 0x7fb898039aa0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0050cf0 tx=0x7fb8a0097ef0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89ffff640 1 -- 172.21.15.12:0/826518378 shutdown_connections 2024-04-27T07:04:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89ffff640 1 --2- 172.21.15.12:0/826518378 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980395f0 0x7fb898039aa0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89ffff640 1 --2- 172.21.15.12:0/826518378 >> v2:172.21.15.12:3300/0 conn(0x7fb8980097d0 0x7fb898008f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89ffff640 1 -- 172.21.15.12:0/826518378 >> 172.21.15.12:0/826518378 conn(0x7fb89803c6b0 msgr2=0x7fb89803cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89ffff640 1 -- 172.21.15.12:0/826518378 shutdown_connections 2024-04-27T07:04:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89ffff640 1 -- 172.21.15.12:0/826518378 wait complete. 2024-04-27T07:04:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89ffff640 1 Processor -- start 2024-04-27T07:04:27.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.613+0000 7fb89ffff640 1 -- start start 2024-04-27T07:04:27.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89ffff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980097d0 0x7fb8980c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8980395f0 0x7fb898164d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8980cc6c0 0x7fb8981652b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8980395f0 0x7fb898164d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8980395f0 0x7fb898164d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55528/0 (socket says 172.21.15.12:55528) 2024-04-27T07:04:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89e7fc640 1 -- 172.21.15.12:0/3715859157 learned_addr learned my addr 172.21.15.12:0/3715859157 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89effd640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980097d0 0x7fb8980c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89ffff640 1 -- 172.21.15.12:0/3715859157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8980d96a0 con 0x7fb8980395f0 2024-04-27T07:04:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89ffff640 1 -- 172.21.15.12:0/3715859157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb898036a50 con 0x7fb8980cc6c0 2024-04-27T07:04:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89ffff640 1 -- 172.21.15.12:0/3715859157 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fb8980d9820 con 0x7fb8980097d0 2024-04-27T07:04:27.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89f7fe640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8980cc6c0 0x7fb8981652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89e7fc640 1 -- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980097d0 msgr2=0x7fb8980c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89e7fc640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980097d0 0x7fb8980c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89e7fc640 1 -- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8980cc6c0 msgr2=0x7fb8981652b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89e7fc640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8980cc6c0 0x7fb8981652b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89e7fc640 1 -- 172.21.15.12:0/3715859157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8980c7790 con 0x7fb8980395f0 2024-04-27T07:04:27.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89e7fc640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8980395f0 0x7fb898164d70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb89000c400 tx=0x7fb89000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb8877fe640 1 -- 172.21.15.12:0/3715859157 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890017070 con 0x7fb8980395f0 2024-04-27T07:04:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb8877fe640 1 -- 172.21.15.12:0/3715859157 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb890005cf0 con 0x7fb8980395f0 2024-04-27T07:04:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb8877fe640 1 -- 172.21.15.12:0/3715859157 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8900072d0 con 0x7fb8980395f0 2024-04-27T07:04:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89ffff640 1 -- 172.21.15.12:0/3715859157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8980c74b0 con 0x7fb8980395f0 2024-04-27T07:04:27.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7fb89ffff640 1 -- 172.21.15.12:0/3715859157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8980c4080 con 0x7fb8980395f0 2024-04-27T07:04:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7f3d160e5640 1 Processor -- start 2024-04-27T07:04:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7f3d160e5640 1 -- start start 2024-04-27T07:04:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7f3d160e5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d1009aa70 0x7f3d100b5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7f3d160e5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d1008b180 con 0x7f3d10157070 2024-04-27T07:04:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7f3d160e5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d100ae5b0 con 0x7f3d1009ae50 2024-04-27T07:04:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7f3d160e5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3d1009b900 con 0x7f3d1009aa70 2024-04-27T07:04:27.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7f3d0ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3d10157070 0x7f3d10157500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59626/0 (socket says 172.21.15.12:59626) 2024-04-27T07:04:27.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7f3d0ffff640 1 -- 172.21.15.12:0/1517255481 learned_addr learned my addr 172.21.15.12:0/1517255481 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:27.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.617+0000 7f3d0f7fe640 1 --2- 172.21.15.12:0/1517255481 >> v2:172.21.15.53:3300/0 conn(0x7f3d1009aa70 0x7f3d100b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.621+0000 7f3d0f7fe640 1 -- 172.21.15.12:0/1517255481 >> v1:172.21.15.53:6789/0 conn(0x7f3d1009ae50 legacy=0x7f3d100af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.621+0000 7f3d0f7fe640 1 -- 172.21.15.12:0/1517255481 >> v1:172.21.15.12:6789/0 conn(0x7f3d10157070 legacy=0x7f3d10157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.621+0000 7f3d0f7fe640 1 -- 172.21.15.12:0/1517255481 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d100b0320 con 0x7f3d1009aa70 2024-04-27T07:04:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.621+0000 7fb8877fe640 1 -- 172.21.15.12:0/3715859157 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb89001a070 con 0x7fb8980395f0 2024-04-27T07:04:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.621+0000 7fb89ffff640 1 -- 172.21.15.12:0/3715859157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb898005ee0 con 0x7fb8980395f0 2024-04-27T07:04:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.621+0000 7fb8877fe640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb88c0683d0 0x7fb88c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.621+0000 7fb8877fe640 1 -- 172.21.15.12:0/3715859157 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb890085c40 con 0x7fb8980395f0 2024-04-27T07:04:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.621+0000 7fb89effd640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb88c0683d0 0x7fb88c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.621+0000 7fb89effd640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb88c0683d0 0x7fb88c06a890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb894009990 tx=0x7fb894009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.625+0000 7f3d0f7fe640 1 --2- 172.21.15.12:0/1517255481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d1009aa70 0x7f3d100b5700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3d04006d20 tx=0x7f3d040201e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71c0e9e21973fe41 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.625+0000 7f3d0e7fc640 1 -- 172.21.15.12:0/1517255481 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d040a8650 con 0x7f3d1009aa70 2024-04-27T07:04:27.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.625+0000 7f3d0e7fc640 1 -- 172.21.15.12:0/1517255481 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d040a87f0 con 0x7f3d1009aa70 2024-04-27T07:04:27.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.625+0000 7f3d160e5640 1 -- 172.21.15.12:0/1517255481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d1009aa70 msgr2=0x7f3d100b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.625+0000 7f3d0e7fc640 1 -- 172.21.15.12:0/1517255481 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d040a8650 con 0x7f3d1009aa70 2024-04-27T07:04:27.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.625+0000 7f3d160e5640 1 --2- 172.21.15.12:0/1517255481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d1009aa70 0x7f3d100b5700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3d04006d20 tx=0x7f3d040201e0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.629+0000 7f3d160e5640 1 -- 172.21.15.12:0/1517255481 shutdown_connections 2024-04-27T07:04:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.629+0000 7f3d160e5640 1 --2- 172.21.15.12:0/1517255481 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d1009aa70 0x7f3d100b5700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.629+0000 7f3d160e5640 1 -- 172.21.15.12:0/1517255481 >> 172.21.15.12:0/1517255481 conn(0x7f3d100b5e40 msgr2=0x7f3d100b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.629+0000 7f3d160e5640 1 -- 172.21.15.12:0/1517255481 shutdown_connections 2024-04-27T07:04:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.629+0000 7fb8877fe640 1 -- 172.21.15.12:0/3715859157 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb8980c7790 con 0x7fb8980395f0 2024-04-27T07:04:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.629+0000 7f3d160e5640 1 -- 172.21.15.12:0/1517255481 wait complete. 2024-04-27T07:04:27.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d160e5640 1 Processor -- start 2024-04-27T07:04:27.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f04830fb640 1 Processor -- start 2024-04-27T07:04:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f603d2f5640 1 Processor -- start 2024-04-27T07:04:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f603d2f5640 1 -- start start 2024-04-27T07:04:27.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f04830fb640 1 -- start start 2024-04-27T07:04:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f603d2f5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6030004a40 0x7f60300387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f603d2f5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60300c9050 0x7f60300c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f603d2f5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60300c9460 con 0x7f60300363a0 2024-04-27T07:04:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f603d2f5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f60300c8bc0 con 0x7f6030004a40 2024-04-27T07:04:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f603d2f5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f60300c8850 con 0x7f60300c9050 2024-04-27T07:04:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d160e5640 1 -- start start 2024-04-27T07:04:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d160e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1009aa70 0x7f3d101f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d160e5640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3d1009ae50 0x7f3d100b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d160e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d10157070 0x7f3d10089420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d160e5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1006e770 con 0x7f3d1009aa70 2024-04-27T07:04:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d160e5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10163070 con 0x7f3d10157070 2024-04-27T07:04:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d160e5640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3d101631f0 con 0x7f3d1009ae50 2024-04-27T07:04:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d10157070 0x7f3d10089420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d10157070 0x7f3d10089420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:39016/0 (socket says 172.21.15.12:39016) 2024-04-27T07:04:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0ffff640 1 -- 172.21.15.12:0/1855938224 learned_addr learned my addr 172.21.15.12:0/1855938224 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0effd640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3d1009ae50 0x7f3d100b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f603caf4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60300c9050 0x7f60300c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f603caf4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f60300c9050 0x7f60300c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:39028/0 (socket says 172.21.15.12:39028) 2024-04-27T07:04:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0ffff640 1 -- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3d1009ae50 msgr2=0x7f3d100b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0ffff640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3d1009ae50 0x7f3d100b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0ffff640 1 -- 172.21.15.12:0/1855938224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1009aa70 msgr2=0x7f3d101f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0f7fe640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1009aa70 0x7f3d101f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0ffff640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1009aa70 0x7f3d101f0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0ffff640 1 -- 172.21.15.12:0/1855938224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d100b5700 con 0x7f3d10157070 2024-04-27T07:04:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0f7fe640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1009aa70 0x7f3d101f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f04830fb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f047c09acd0 0x7f047c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f04830fb640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f047c09f250 0x7f047c145e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f04830fb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f047c1463b0 0x7f047c13f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f04830fb640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f047c06e7a0 con 0x7f047c1463b0 2024-04-27T07:04:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f04830fb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f047c146800 con 0x7f047c09acd0 2024-04-27T07:04:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f04830fb640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f047c09b0b0 con 0x7f047c09f250 2024-04-27T07:04:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f0481698640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f047c1463b0 0x7f047c13f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f0481698640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f047c1463b0 0x7f047c13f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55562/0 (socket says 172.21.15.12:55562) 2024-04-27T07:04:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f0481698640 1 -- 172.21.15.12:0/3139903800 learned_addr learned my addr 172.21.15.12:0/3139903800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f603caf4640 1 -- 172.21.15.12:0/1603914197 learned_addr learned my addr 172.21.15.12:0/1603914197 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f6037fff640 1 --2- 172.21.15.12:0/1603914197 >> v2:172.21.15.12:3300/0 conn(0x7f6030004a40 0x7f60300387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f0480e97640 1 --2- 172.21.15.12:0/3139903800 >> v2:172.21.15.53:3300/0 conn(0x7f047c09acd0 0x7f047c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3d0ffff640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d10157070 0x7f3d10089420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000c780 tx=0x7f3d0000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f3cf7fff640 1 -- 172.21.15.12:0/1855938224 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00018070 con 0x7f3d10157070 2024-04-27T07:04:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f3cf7fff640 1 -- 172.21.15.12:0/1855938224 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d000072d0 con 0x7f3d10157070 2024-04-27T07:04:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f3cf7fff640 1 -- 172.21.15.12:0/1855938224 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000047f0 con 0x7f3d10157070 2024-04-27T07:04:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f3d160e5640 1 -- 172.21.15.12:0/1855938224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d100b3320 con 0x7f3d10157070 2024-04-27T07:04:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f3d160e5640 1 -- 172.21.15.12:0/1855938224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d101f17a0 con 0x7f3d10157070 2024-04-27T07:04:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f3d160e5640 1 -- 172.21.15.12:0/1855938224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d10068c20 con 0x7f3d10157070 2024-04-27T07:04:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f3cf7fff640 1 -- 172.21.15.12:0/1855938224 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3d0000a640 con 0x7f3d10157070 2024-04-27T07:04:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f0481698640 1 -- 172.21.15.12:0/3139903800 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f047c09f250 msgr2=0x7f047c145e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-27T07:04:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f0481698640 1 --2- 172.21.15.12:0/3139903800 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f047c09f250 0x7f047c145e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f0481698640 1 -- 172.21.15.12:0/3139903800 >> v2:172.21.15.53:3300/0 conn(0x7f047c09acd0 msgr2=0x7f047c06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f0481698640 1 --2- 172.21.15.12:0/3139903800 >> v2:172.21.15.53:3300/0 conn(0x7f047c09acd0 0x7f047c06e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f0481698640 1 -- 172.21.15.12:0/3139903800 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c09b9d0 con 0x7f047c1463b0 2024-04-27T07:04:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f0480e97640 1 --2- 172.21.15.12:0/3139903800 >> v2:172.21.15.53:3300/0 conn(0x7f047c09acd0 0x7f047c06e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f0481698640 1 --2- 172.21.15.12:0/3139903800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c1463b0 0x7f047c13f7f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f047c070140 tx=0x7f047001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce0ed8e329765a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f3cf7fff640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cfc068380 0x7f3cfc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f3cf7fff640 1 -- 172.21.15.12:0/1855938224 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d00086020 con 0x7f3d10157070 2024-04-27T07:04:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.633+0000 7f6036ffd640 1 -- 172.21.15.12:0/1603914197 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4282926367 0 0) 0x7f60300c9460 con 0x7f60300363a0 2024-04-27T07:04:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f047b7fe640 1 -- 172.21.15.12:0/3139903800 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04700b0070 con 0x7f047c1463b0 2024-04-27T07:04:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f047b7fe640 1 -- 172.21.15.12:0/3139903800 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04700ab600 con 0x7f047c1463b0 2024-04-27T07:04:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f6036ffd640 1 -- 172.21.15.12:0/1603914197 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6028003530 con 0x7f60300363a0 2024-04-27T07:04:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f6037fff640 1 -- 172.21.15.12:0/1603914197 >> v2:172.21.15.53:3300/0 conn(0x7f60300c9050 msgr2=0x7f60300c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f6037fff640 1 --2- 172.21.15.12:0/1603914197 >> v2:172.21.15.53:3300/0 conn(0x7f60300c9050 0x7f60300c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f6037fff640 1 -- 172.21.15.12:0/1603914197 >> v1:172.21.15.53:6789/0 conn(0x7f60300363a0 legacy=0x7f6030036df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.637+0000 7f6037fff640 1 -- 172.21.15.12:0/1603914197 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6030037ec0 con 0x7f6030004a40 2024-04-27T07:04:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f3d0f7fe640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cfc068380 0x7f3cfc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 -- 172.21.15.12:0/3139903800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c1463b0 msgr2=0x7f047c13f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f047b7fe640 1 -- 172.21.15.12:0/3139903800 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04700b0070 con 0x7f047c1463b0 2024-04-27T07:04:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 --2- 172.21.15.12:0/3139903800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c1463b0 0x7f047c13f7f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f047c070140 tx=0x7f047001c4b0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f3d0f7fe640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cfc068380 0x7f3cfc06a840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3d040b1000 tx=0x7f3d040aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 -- 172.21.15.12:0/3139903800 shutdown_connections 2024-04-27T07:04:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 --2- 172.21.15.12:0/3139903800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c1463b0 0x7f047c13f7f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 --2- 172.21.15.12:0/3139903800 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f047c09f250 0x7f047c145e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 --2- 172.21.15.12:0/3139903800 >> v2:172.21.15.53:3300/0 conn(0x7f047c09acd0 0x7f047c06e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 -- 172.21.15.12:0/3139903800 >> 172.21.15.12:0/3139903800 conn(0x7f047c0b5e40 msgr2=0x7f047c0b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 -- 172.21.15.12:0/3139903800 shutdown_connections 2024-04-27T07:04:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 -- 172.21.15.12:0/3139903800 wait complete. 2024-04-27T07:04:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f6037fff640 1 --2- 172.21.15.12:0/1603914197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030004a40 0x7f60300387e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f60300c8d40 tx=0x7f6038097f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69241b8bde8b139 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 Processor -- start 2024-04-27T07:04:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f6036ffd640 1 -- 172.21.15.12:0/1603914197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038120a60 con 0x7f6030004a40 2024-04-27T07:04:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f6036ffd640 1 -- 172.21.15.12:0/1603914197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6038120c40 con 0x7f6030004a40 2024-04-27T07:04:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f6036ffd640 1 -- 172.21.15.12:0/1603914197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038128640 con 0x7f6030004a40 2024-04-27T07:04:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.641+0000 7f04830fb640 1 -- start start 2024-04-27T07:04:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04830fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c09acd0 0x7f047c0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04830fb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c09f250 0x7f047c142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04830fb640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f047c1463b0 0x7f047c0b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04830fb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f047c149b30 con 0x7f047c09f250 2024-04-27T07:04:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04830fb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1499b0 con 0x7f047c09acd0 2024-04-27T07:04:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04830fb640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f047c149cb0 con 0x7f047c1463b0 2024-04-27T07:04:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 -- 172.21.15.12:0/1603914197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030004a40 msgr2=0x7f60300387e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 --2- 172.21.15.12:0/1603914197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030004a40 0x7f60300387e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f60300c8d40 tx=0x7f6038097f10 comp rx=0 tx=0).stop 2024-04-27T07:04:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f047bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c09f250 0x7f047c142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f0480e97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c09acd0 0x7f047c0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f047bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c09f250 0x7f047c142d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55572/0 (socket says 172.21.15.12:55572) 2024-04-27T07:04:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f0480e97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c09acd0 0x7f047c0b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:39036/0 (socket says 172.21.15.12:39036) 2024-04-27T07:04:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f0480e97640 1 -- 172.21.15.12:0/4137722625 learned_addr learned my addr 172.21.15.12:0/4137722625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 -- 172.21.15.12:0/1603914197 shutdown_connections 2024-04-27T07:04:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 --2- 172.21.15.12:0/1603914197 >> v2:172.21.15.53:3300/0 conn(0x7f60300c9050 0x7f60300c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 --2- 172.21.15.12:0/1603914197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030004a40 0x7f60300387e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 -- 172.21.15.12:0/1603914197 >> 172.21.15.12:0/1603914197 conn(0x7f603003ace0 msgr2=0x7f603003b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 -- 172.21.15.12:0/1603914197 shutdown_connections 2024-04-27T07:04:27.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 -- 172.21.15.12:0/1603914197 wait complete. 2024-04-27T07:04:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f0481698640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f047c1463b0 0x7f047c0b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f047bfff640 1 -- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f047c1463b0 msgr2=0x7f047c0b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f047bfff640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f047c1463b0 0x7f047c0b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f047bfff640 1 -- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c09acd0 msgr2=0x7f047c0b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f047bfff640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c09acd0 0x7f047c0b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f047bfff640 1 -- 172.21.15.12:0/4137722625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c1432d0 con 0x7f047c09f250 2024-04-27T07:04:27.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f047bfff640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c09f250 0x7f047c142d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f047400be50 tx=0x7f0474009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04797fa640 1 -- 172.21.15.12:0/4137722625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474019070 con 0x7f047c09f250 2024-04-27T07:04:27.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04830fb640 1 -- 172.21.15.12:0/4137722625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047c0b15e0 con 0x7f047c09f250 2024-04-27T07:04:27.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04797fa640 1 -- 172.21.15.12:0/4137722625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0474014440 con 0x7f047c09f250 2024-04-27T07:04:27.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04830fb640 1 -- 172.21.15.12:0/4137722625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047c1421b0 con 0x7f047c09f250 2024-04-27T07:04:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 Processor -- start 2024-04-27T07:04:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 -- start start 2024-04-27T07:04:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04797fa640 1 -- 172.21.15.12:0/4137722625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474019070 con 0x7f047c09f250 2024-04-27T07:04:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f04830fb640 1 -- 172.21.15.12:0/4137722625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f047c068c20 con 0x7f047c09f250 2024-04-27T07:04:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6030004a40 0x7f6030169030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60300363a0 0x7f60300c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60300c9050 0x7f6030033920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f60300cd7e0 con 0x7f60300c9050 2024-04-27T07:04:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f60300cd660 con 0x7f60300363a0 2024-04-27T07:04:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f603d2f5640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f60300cd960 con 0x7f6030004a40 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f60377fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60300363a0 0x7f60300c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.645+0000 7f6037fff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6030004a40 0x7f6030169030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f6037fff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6030004a40 0x7f6030169030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:56786/0 (socket says 172.21.15.12:56786) 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f60377fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60300363a0 0x7f60300c31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:39046/0 (socket says 172.21.15.12:39046) 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f6037fff640 1 -- 172.21.15.12:0/2332515903 learned_addr learned my addr 172.21.15.12:0/2332515903 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f603caf4640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60300c9050 0x7f6030033920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f60377fe640 1 -- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6030004a40 msgr2=0x7f6030169030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f60377fe640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6030004a40 0x7f6030169030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f60377fe640 1 -- 172.21.15.12:0/2332515903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60300c9050 msgr2=0x7f6030033920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f60377fe640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60300c9050 0x7f6030033920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f60377fe640 1 -- 172.21.15.12:0/2332515903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60300c3700 con 0x7f60300363a0 2024-04-27T07:04:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f60377fe640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60300363a0 0x7f60300c31c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f602c00c430 tx=0x7f602c00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f04797fa640 1 -- 172.21.15.12:0/4137722625 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f047401c070 con 0x7f047c09f250 2024-04-27T07:04:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f04797fa640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0468068580 0x7f046806aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f04797fa640 1 -- 172.21.15.12:0/4137722625 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0474054c60 con 0x7f047c09f250 2024-04-27T07:04:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f0480e97640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0468068580 0x7f046806aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.649+0000 7f0480e97640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0468068580 0x7f046806aa40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f046c007a70 tx=0x7f046c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.653+0000 7f3cf7fff640 1 -- 172.21.15.12:0/1855938224 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3d00021020 con 0x7f3d10157070 2024-04-27T07:04:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.653+0000 7f6034ff9640 1 -- 172.21.15.12:0/2332515903 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c017070 con 0x7f60300363a0 2024-04-27T07:04:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.653+0000 7f6034ff9640 1 -- 172.21.15.12:0/2332515903 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f602c0052d0 con 0x7f60300363a0 2024-04-27T07:04:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.653+0000 7f603d2f5640 1 -- 172.21.15.12:0/2332515903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60300c39b0 con 0x7f60300363a0 2024-04-27T07:04:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.653+0000 7f6034ff9640 1 -- 172.21.15.12:0/2332515903 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0054c0 con 0x7f60300363a0 2024-04-27T07:04:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.653+0000 7f603d2f5640 1 -- 172.21.15.12:0/2332515903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6030033400 con 0x7f60300363a0 2024-04-27T07:04:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.653+0000 7f603d2f5640 1 -- 172.21.15.12:0/2332515903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6030005720 con 0x7f60300363a0 2024-04-27T07:04:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.653+0000 7f6034ff9640 1 -- 172.21.15.12:0/2332515903 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f602c01a070 con 0x7f60300363a0 2024-04-27T07:04:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.653+0000 7f6034ff9640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f601c068450 0x7f601c06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.653+0000 7f6034ff9640 1 -- 172.21.15.12:0/2332515903 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f602c086100 con 0x7f60300363a0 2024-04-27T07:04:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.657+0000 7f6037fff640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f601c068450 0x7f601c06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.657+0000 7f04797fa640 1 -- 172.21.15.12:0/4137722625 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0474056bc0 con 0x7f047c09f250 2024-04-27T07:04:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.657+0000 7f6037fff640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f601c068450 0x7f601c06a910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60380656d0 tx=0x7f6038122070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.665+0000 7fcd7adad640 1 Processor -- start 2024-04-27T07:04:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.665+0000 7fcd7adad640 1 -- start start 2024-04-27T07:04:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.665+0000 7f6034ff9640 1 -- 172.21.15.12:0/2332515903 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f602c021020 con 0x7f60300363a0 2024-04-27T07:04:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd7adad640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 0x7fcd741453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd7adad640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcd740b8ec0 0x7fcd741458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd7adad640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd740b77a0 con 0x7fcd740b5b00 2024-04-27T07:04:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd7adad640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fcd740ba490 con 0x7fcd740b8ec0 2024-04-27T07:04:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd7adad640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fcd740b5ff0 con 0x7fcd740b73c0 2024-04-27T07:04:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd73fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcd740b5b00 0x7fcd740b87b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59634/0 (socket says 172.21.15.12:59634) 2024-04-27T07:04:27.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd73fff640 1 -- 172.21.15.12:0/1180515219 learned_addr learned my addr 172.21.15.12:0/1180515219 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:27.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd78b49640 1 --2- 172.21.15.12:0/1180515219 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 0x7fcd741453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd7934a640 1 --2- 172.21.15.12:0/1180515219 >> v2:172.21.15.53:3300/0 conn(0x7fcd740b8ec0 0x7fcd741458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd78b49640 1 -- 172.21.15.12:0/1180515219 >> v2:172.21.15.53:3300/0 conn(0x7fcd740b8ec0 msgr2=0x7fcd741458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:27.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd78b49640 1 --2- 172.21.15.12:0/1180515219 >> v2:172.21.15.53:3300/0 conn(0x7fcd740b8ec0 0x7fcd741458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd78b49640 1 -- 172.21.15.12:0/1180515219 >> v1:172.21.15.12:6789/0 conn(0x7fcd740b5b00 legacy=0x7fcd740b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.669+0000 7fcd78b49640 1 -- 172.21.15.12:0/1180515219 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd740b62d0 con 0x7fcd740b73c0 2024-04-27T07:04:27.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.673+0000 7fcd7934a640 1 --2- 172.21.15.12:0/1180515219 >> v2:172.21.15.53:3300/0 conn(0x7fcd740b8ec0 0x7fcd741458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:04:27.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.673+0000 7fcd78b49640 1 --2- 172.21.15.12:0/1180515219 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 0x7fcd741453b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd600097c0 tx=0x7fcd6001e660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74eae86c00d83364 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.673+0000 7fcd737fe640 1 -- 172.21.15.12:0/1180515219 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd600ad660 con 0x7fcd740b73c0 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.673+0000 7fcd737fe640 1 -- 172.21.15.12:0/1180515219 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd600b2070 con 0x7fcd740b73c0 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.673+0000 7fcd737fe640 1 -- 172.21.15.12:0/1180515219 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd600ada60 con 0x7fcd740b73c0 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 -- 172.21.15.12:0/1180515219 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 msgr2=0x7fcd741453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 --2- 172.21.15.12:0/1180515219 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 0x7fcd741453b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd600097c0 tx=0x7fcd6001e660 comp rx=0 tx=0).stop 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 -- 172.21.15.12:0/1180515219 shutdown_connections 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 --2- 172.21.15.12:0/1180515219 >> v2:172.21.15.53:3300/0 conn(0x7fcd740b8ec0 0x7fcd741458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 --2- 172.21.15.12:0/1180515219 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 0x7fcd741453b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 -- 172.21.15.12:0/1180515219 >> 172.21.15.12:0/1180515219 conn(0x7fcd740ba6d0 msgr2=0x7fcd740baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 -- 172.21.15.12:0/1180515219 shutdown_connections 2024-04-27T07:04:27.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 -- 172.21.15.12:0/1180515219 wait complete. 2024-04-27T07:04:27.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 Processor -- start 2024-04-27T07:04:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 -- start start 2024-04-27T07:04:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd740b5b00 0x7fcd7409b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 0x7fcd74141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740b8ec0 0x7fcd741f1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7406c720 con 0x7fcd740b8ec0 2024-04-27T07:04:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7406c5a0 con 0x7fcd740b5b00 2024-04-27T07:04:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.677+0000 7fcd7adad640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7406c8a0 con 0x7fcd740b73c0 2024-04-27T07:04:27.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7934a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740b8ec0 0x7fcd741f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd78b49640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd740b5b00 0x7fcd7409b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7934a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740b8ec0 0x7fcd741f1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55590/0 (socket says 172.21.15.12:55590) 2024-04-27T07:04:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7934a640 1 -- 172.21.15.12:0/1879055241 learned_addr learned my addr 172.21.15.12:0/1879055241 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd73fff640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 0x7fcd74141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7934a640 1 -- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 msgr2=0x7fcd74141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7934a640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 0x7fcd74141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7934a640 1 -- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd740b5b00 msgr2=0x7fcd7409b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7934a640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd740b5b00 0x7fcd7409b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7934a640 1 -- 172.21.15.12:0/1879055241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd740b1970 con 0x7fcd740b8ec0 2024-04-27T07:04:27.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd73fff640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 0x7fcd74141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:04:27.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd78b49640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd740b5b00 0x7fcd7409b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:04:27.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7934a640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740b8ec0 0x7fcd741f1150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00efc0 tx=0x7fcd6c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd717fa640 1 -- 172.21.15.12:0/1879055241 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c018030 con 0x7fcd740b8ec0 2024-04-27T07:04:27.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7adad640 1 -- 172.21.15.12:0/1879055241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7409bf50 con 0x7fcd740b8ec0 2024-04-27T07:04:27.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd7adad640 1 -- 172.21.15.12:0/1879055241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd74141f70 con 0x7fcd740b8ec0 2024-04-27T07:04:27.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd717fa640 1 -- 172.21.15.12:0/1879055241 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6c00cde0 con 0x7fcd740b8ec0 2024-04-27T07:04:27.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.681+0000 7fcd717fa640 1 -- 172.21.15.12:0/1879055241 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c003bc0 con 0x7fcd740b8ec0 2024-04-27T07:04:27.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.685+0000 7fcd717fa640 1 -- 172.21.15.12:0/1879055241 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcd6c01b070 con 0x7fcd740b8ec0 2024-04-27T07:04:27.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.685+0000 7fcd717fa640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcd640684c0 0x7fcd6406a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.685+0000 7fcd717fa640 1 -- 172.21.15.12:0/1879055241 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcd6c087290 con 0x7fcd740b8ec0 2024-04-27T07:04:27.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.685+0000 7fcd78b49640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcd640684c0 0x7fcd6406a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.685+0000 7fcd78b49640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcd640684c0 0x7fcd6406a980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd600b1000 tx=0x7fcd600a9000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7fcd7adad640 1 -- 172.21.15.12:0/1879055241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd580052a0 con 0x7fcd740b8ec0 2024-04-27T07:04:27.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fde8a6640 1 Processor -- start 2024-04-27T07:04:27.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fde8a6640 1 -- start start 2024-04-27T07:04:27.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fde8a6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0fd00cc4b0 0x7f0fd00c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fdd0a3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0fd00c95b0 0x7f0fd00c2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59646/0 (socket says 172.21.15.12:59646) 2024-04-27T07:04:27.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fdd8a4640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f0fd0037330 0x7f0fd00c8ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:35532/0 (socket says 172.21.15.12:35532) 2024-04-27T07:04:27.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fdd0a3640 1 -- 172.21.15.12:0/564353547 learned_addr learned my addr 172.21.15.12:0/564353547 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:27.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fde8a6640 1 -- 172.21.15.12:0/564353547 --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fd00c30d0 con 0x7f0fd00c95b0 2024-04-27T07:04:27.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fde8a6640 1 -- 172.21.15.12:0/564353547 --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fd0037710 con 0x7f0fd0037330 2024-04-27T07:04:27.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fde8a6640 1 -- 172.21.15.12:0/564353547 --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd00c89e0 con 0x7f0fd00cc4b0 2024-04-27T07:04:27.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fdc8a2640 1 -- 172.21.15.12:0/564353547 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596738483 0 0) 0x7f0fd0037710 con 0x7f0fd0037330 2024-04-27T07:04:27.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.689+0000 7f0fdc8a2640 1 -- 172.21.15.12:0/564353547 --> v1:172.21.15.53:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fc4003530 con 0x7f0fd0037330 2024-04-27T07:04:27.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.693+0000 7f0fde0a5640 1 --2- 172.21.15.12:0/564353547 >> v2:172.21.15.12:3300/0 conn(0x7f0fd00cc4b0 0x7f0fd00c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.693+0000 7f0fde0a5640 1 -- 172.21.15.12:0/564353547 >> v1:172.21.15.53:6789/0 conn(0x7f0fd0037330 legacy=0x7f0fd00c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.693+0000 7f0fde0a5640 1 -- 172.21.15.12:0/564353547 >> v1:172.21.15.12:6789/0 conn(0x7f0fd00c95b0 legacy=0x7f0fd00c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.693+0000 7f0fde0a5640 1 -- 172.21.15.12:0/564353547 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd002a440 con 0x7f0fd00cc4b0 2024-04-27T07:04:27.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f0fde0a5640 1 --2- 172.21.15.12:0/564353547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fd00cc4b0 0x7f0fd00c2b90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8000b70 tx=0x7f0fd8098060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b379eb8947239f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b9463640 1 Processor -- start 2024-04-27T07:04:27.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b9463640 1 -- start start 2024-04-27T07:04:27.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7fcd717fa640 1 -- 172.21.15.12:0/1879055241 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcd6c022070 con 0x7fcd740b8ec0 2024-04-27T07:04:27.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f0fdc8a2640 1 -- 172.21.15.12:0/564353547 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8128440 con 0x7f0fd00cc4b0 2024-04-27T07:04:27.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f0fdc8a2640 1 -- 172.21.15.12:0/564353547 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd8128620 con 0x7f0fd00cc4b0 2024-04-27T07:04:27.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 -- 172.21.15.12:0/564353547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fd00cc4b0 msgr2=0x7f0fd00c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 --2- 172.21.15.12:0/564353547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fd00cc4b0 0x7f0fd00c2b90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8000b70 tx=0x7f0fd8098060 comp rx=0 tx=0).stop 2024-04-27T07:04:27.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 -- 172.21.15.12:0/564353547 shutdown_connections 2024-04-27T07:04:27.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 --2- 172.21.15.12:0/564353547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fd00cc4b0 0x7f0fd00c2b90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 -- 172.21.15.12:0/564353547 >> 172.21.15.12:0/564353547 conn(0x7f0fd003b120 msgr2=0x7f0fd003b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b9463640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19ac038e10 0x7f19ac039770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b9463640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19ac0c40b0 0x7f19ac0c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b9463640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19ac0cb220 con 0x7f19ac037340 2024-04-27T07:04:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b9463640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f19ac0c4a60 con 0x7f19ac0c40b0 2024-04-27T07:04:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b9463640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f19ac037720 con 0x7f19ac038e10 2024-04-27T07:04:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b8c62640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19ac0c40b0 0x7f19ac0c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b8c62640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19ac0c40b0 0x7f19ac0c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:39084/0 (socket says 172.21.15.12:39084) 2024-04-27T07:04:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b8c62640 1 -- 172.21.15.12:0/1979455502 learned_addr learned my addr 172.21.15.12:0/1979455502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b37fe640 1 --2- 172.21.15.12:0/1979455502 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19ac038e10 0x7f19ac039770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b8c62640 1 -- 172.21.15.12:0/1979455502 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19ac038e10 msgr2=0x7f19ac039770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b8c62640 1 --2- 172.21.15.12:0/1979455502 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19ac038e10 0x7f19ac039770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.697+0000 7f19b8c62640 1 -- 172.21.15.12:0/1979455502 >> v1:172.21.15.12:6789/0 conn(0x7f19ac037340 legacy=0x7f19ac0cab10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T07:04:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f19b8c62640 1 -- 172.21.15.12:0/1979455502 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ac039dd0 con 0x7f19ac0c40b0 2024-04-27T07:04:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 -- 172.21.15.12:0/564353547 shutdown_connections 2024-04-27T07:04:27.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 -- 172.21.15.12:0/564353547 wait complete. 2024-04-27T07:04:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 Processor -- start 2024-04-27T07:04:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f19b8c62640 1 --2- 172.21.15.12:0/1979455502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19ac0c40b0 0x7f19ac0c4520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0396c0 tx=0x7f19b4097e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ae02253b68325e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f19b2ffd640 1 -- 172.21.15.12:0/1979455502 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4120bf0 con 0x7f19ac0c40b0 2024-04-27T07:04:27.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f19b2ffd640 1 -- 172.21.15.12:0/1979455502 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b4120d90 con 0x7f19ac0c40b0 2024-04-27T07:04:27.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 -- start start 2024-04-27T07:04:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fd0037330 0x7f0fd00d9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fd00c95b0 0x7f0fd0168730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.701+0000 7f0fde8a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd00b82c0 0x7f0fd0034160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fde8a6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd00d6d10 con 0x7f0fd00c95b0 2024-04-27T07:04:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fde8a6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd00d6b90 con 0x7f0fd00b82c0 2024-04-27T07:04:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fde8a6640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd00d6e90 con 0x7f0fd0037330 2024-04-27T07:04:27.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd8a4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fd0037330 0x7f0fd00d9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd0a3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fd00c95b0 0x7f0fd0168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd0a3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fd00c95b0 0x7f0fd0168730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55594/0 (socket says 172.21.15.12:55594) 2024-04-27T07:04:27.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd0a3640 1 -- 172.21.15.12:0/1911208134 learned_addr learned my addr 172.21.15.12:0/1911208134 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd8a4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fd0037330 0x7f0fd00d9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:56820/0 (socket says 172.21.15.12:56820) 2024-04-27T07:04:27.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fde0a5640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd00b82c0 0x7f0fd0034160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f19b9463640 1 -- 172.21.15.12:0/1979455502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19ac0c40b0 msgr2=0x7f19ac0c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f19b9463640 1 --2- 172.21.15.12:0/1979455502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19ac0c40b0 0x7f19ac0c4520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0396c0 tx=0x7f19b4097e90 comp rx=0 tx=0).stop 2024-04-27T07:04:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd8a4640 1 -- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd00b82c0 msgr2=0x7f0fd0034160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd8a4640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd00b82c0 0x7f0fd0034160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd8a4640 1 -- 172.21.15.12:0/1911208134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fd00c95b0 msgr2=0x7f0fd0168730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd8a4640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fd00c95b0 0x7f0fd0168730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd8a4640 1 -- 172.21.15.12:0/1911208134 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd016bdd0 con 0x7f0fd0037330 2024-04-27T07:04:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f19b9463640 1 -- 172.21.15.12:0/1979455502 shutdown_connections 2024-04-27T07:04:27.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f19b9463640 1 --2- 172.21.15.12:0/1979455502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19ac0c40b0 0x7f19ac0c4520 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f19b9463640 1 --2- 172.21.15.12:0/1979455502 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19ac038e10 0x7f19ac039770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f19b9463640 1 -- 172.21.15.12:0/1979455502 >> 172.21.15.12:0/1979455502 conn(0x7f19ac03c6b0 msgr2=0x7f19ac03cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fdd8a4640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fd0037330 0x7f0fd00d9d10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400c3a0 tx=0x7f0fd400c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fce7fc640 1 -- 172.21.15.12:0/1911208134 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd40074e0 con 0x7f0fd0037330 2024-04-27T07:04:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fce7fc640 1 -- 172.21.15.12:0/1911208134 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd4007680 con 0x7f0fd0037330 2024-04-27T07:04:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fce7fc640 1 -- 172.21.15.12:0/1911208134 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4004a10 con 0x7f0fd0037330 2024-04-27T07:04:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fde8a6640 1 -- 172.21.15.12:0/1911208134 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd00b9280 con 0x7f0fd0037330 2024-04-27T07:04:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fde8a6640 1 -- 172.21.15.12:0/1911208134 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd00b8ee0 con 0x7f0fd0037330 2024-04-27T07:04:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.705+0000 7f0fde8a6640 1 -- 172.21.15.12:0/1911208134 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc00052a0 con 0x7f0fd0037330 2024-04-27T07:04:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.709+0000 7f0fce7fc640 1 -- 172.21.15.12:0/1911208134 <== mon.2 v2:172.21.15.53:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0fd401a020 con 0x7f0fd0037330 2024-04-27T07:04:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.709+0000 7f0fce7fc640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fbc068380 0x7f0fbc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.709+0000 7f0fce7fc640 1 -- 172.21.15.12:0/1911208134 <== mon.2 v2:172.21.15.53:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0fd4086690 con 0x7f0fd0037330 2024-04-27T07:04:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.709+0000 7f0fdd0a3640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fbc068380 0x7f0fbc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.709+0000 7f0fdd0a3640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fbc068380 0x7f0fbc06a840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00c2a50 tx=0x7f0fc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b9463640 1 -- 172.21.15.12:0/1979455502 shutdown_connections 2024-04-27T07:04:27.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b9463640 1 -- 172.21.15.12:0/1979455502 wait complete. 2024-04-27T07:04:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b9463640 1 Processor -- start 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b9463640 1 -- start start 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b9463640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19ac037340 0x7f19ac0c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b9463640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19ac038e10 0x7f19ac0bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b9463640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19ac0c40b0 0x7f19ac0c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b9463640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac0d7480 con 0x7f19ac038e10 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b9463640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac03a5b0 con 0x7f19ac037340 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b9463640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f19ac0d7600 con 0x7f19ac0c40b0 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19ac038e10 0x7f19ac0bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19ac038e10 0x7f19ac0bea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55606/0 (socket says 172.21.15.12:55606) 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b37fe640 1 -- 172.21.15.12:0/2298245564 learned_addr learned my addr 172.21.15.12:0/2298245564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b3fff640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19ac037340 0x7f19ac0c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b37fe640 1 -- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19ac0c40b0 msgr2=0x7f19ac0c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b37fe640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19ac0c40b0 0x7f19ac0c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b37fe640 1 -- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19ac037340 msgr2=0x7f19ac0c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b37fe640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19ac037340 0x7f19ac0c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b37fe640 1 -- 172.21.15.12:0/2298245564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ac0c2280 con 0x7f19ac038e10 2024-04-27T07:04:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.713+0000 7f19b3fff640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19ac037340 0x7f19ac0c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:04:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.717+0000 7f19b37fe640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19ac038e10 0x7f19ac0bea70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f19a800e930 tx=0x7f19a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.717+0000 7f19b0ff9640 1 -- 172.21.15.12:0/2298245564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8017070 con 0x7f19ac038e10 2024-04-27T07:04:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.717+0000 7f19b0ff9640 1 -- 172.21.15.12:0/2298245564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a800ca00 con 0x7f19ac038e10 2024-04-27T07:04:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.717+0000 7f19b0ff9640 1 -- 172.21.15.12:0/2298245564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a800cdb0 con 0x7f19ac038e10 2024-04-27T07:04:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.717+0000 7f19b9463640 1 -- 172.21.15.12:0/2298245564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ac0c2530 con 0x7f19ac038e10 2024-04-27T07:04:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.717+0000 7f19b9463640 1 -- 172.21.15.12:0/2298245564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ac0c2af0 con 0x7f19ac038e10 2024-04-27T07:04:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.717+0000 7f19b9463640 1 -- 172.21.15.12:0/2298245564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ac0c7cd0 con 0x7f19ac038e10 2024-04-27T07:04:27.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.721+0000 7f0fce7fc640 1 -- 172.21.15.12:0/1911208134 <== mon.2 v2:172.21.15.53:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0fd4021070 con 0x7f0fd0037330 2024-04-27T07:04:27.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.721+0000 7f19b0ff9640 1 -- 172.21.15.12:0/2298245564 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f19a801a070 con 0x7f19ac038e10 2024-04-27T07:04:27.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.721+0000 7f19b0ff9640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1998068380 0x7f199806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:27.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.721+0000 7f19b0ff9640 1 -- 172.21.15.12:0/2298245564 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f19a8087120 con 0x7f19ac038e10 2024-04-27T07:04:27.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.721+0000 7f19b3fff640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1998068380 0x7f199806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:27.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.721+0000 7f19b3fff640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1998068380 0x7f199806a840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009d50 tx=0x7f19a00090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:27.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.729+0000 7f19b0ff9640 1 -- 172.21.15.12:0/2298245564 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f19a80571c0 con 0x7f19ac038e10 2024-04-27T07:04:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.933+0000 7f22fb91e640 1 -- 172.21.15.12:0/254882703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f22f40af6f0 con 0x7f22f40b58b0 2024-04-27T07:04:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.933+0000 7f22e9ffb640 1 -- 172.21.15.12:0/254882703 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f22e405a080 con 0x7f22f40b58b0 2024-04-27T07:04:27.941 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 -- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22d8068380 msgr2=0x7f22d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22d8068380 0x7f22d806a840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22f0006fd0 tx=0x7f22f0008040 comp rx=0 tx=0).stop 2024-04-27T07:04:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 -- 172.21.15.12:0/254882703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 msgr2=0x7f22f41edc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 0x7f22f41edc60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c450 tx=0x7f22e400c910 comp rx=0 tx=0).stop 2024-04-27T07:04:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 -- 172.21.15.12:0/254882703 shutdown_connections 2024-04-27T07:04:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22d8068380 0x7f22d806a840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22f4147420 0x7f22f413eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22f40b58b0 0x7f22f41edc60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 --2- 172.21.15.12:0/254882703 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22f409aa70 0x7f22f4073710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 -- 172.21.15.12:0/254882703 >> 172.21.15.12:0/254882703 conn(0x7f22f40b5e40 msgr2=0x7f22f414dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 -- 172.21.15.12:0/254882703 shutdown_connections 2024-04-27T07:04:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.937+0000 7f22df7fe640 1 -- 172.21.15.12:0/254882703 wait complete. 2024-04-27T07:04:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.953+0000 7fb89ffff640 1 -- 172.21.15.12:0/3715859157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb8980cb0b0 con 0x7fb8980395f0 2024-04-27T07:04:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.953+0000 7fb8877fe640 1 -- 172.21.15.12:0/3715859157 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb890055c20 con 0x7fb8980395f0 2024-04-27T07:04:27.958 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 -- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb88c0683d0 msgr2=0x7fb88c06a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb88c0683d0 0x7fb88c06a890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb894009990 tx=0x7fb894009110 comp rx=0 tx=0).stop 2024-04-27T07:04:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 -- 172.21.15.12:0/3715859157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8980395f0 msgr2=0x7fb898164d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8980395f0 0x7fb898164d70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb89000c400 tx=0x7fb89000c8c0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 -- 172.21.15.12:0/3715859157 shutdown_connections 2024-04-27T07:04:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb88c0683d0 0x7fb88c06a890 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8980cc6c0 0x7fb8981652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8980395f0 0x7fb898164d70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 --2- 172.21.15.12:0/3715859157 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb8980097d0 0x7fb8980c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 -- 172.21.15.12:0/3715859157 >> 172.21.15.12:0/3715859157 conn(0x7fb89803c6b0 msgr2=0x7fb898038ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.957+0000 7fb8857fa640 1 -- 172.21.15.12:0/3715859157 shutdown_connections 2024-04-27T07:04:27.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.961+0000 7fb8857fa640 1 -- 172.21.15.12:0/3715859157 wait complete. 2024-04-27T07:04:27.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-04-27T07:04:27.965 DEBUG:teuthology.parallel:result is None 2024-04-27T07:04:27.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-04-27T07:04:27.985 DEBUG:teuthology.parallel:result is None 2024-04-27T07:04:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.985+0000 7f04830fb640 1 -- 172.21.15.12:0/4137722625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f047c0b18c0 con 0x7f047c09f250 2024-04-27T07:04:27.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.985+0000 7f04797fa640 1 -- 172.21.15.12:0/4137722625 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f047c0b18c0 con 0x7f047c09f250 2024-04-27T07:04:27.991 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:27.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 -- 172.21.15.12:0/1855938224 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3d101f19e0 con 0x7f3d10157070 2024-04-27T07:04:27.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 -- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0468068580 msgr2=0x7f046806aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0468068580 0x7f046806aa40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f046c007a70 tx=0x7f046c008040 comp rx=0 tx=0).stop 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 -- 172.21.15.12:0/4137722625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c09f250 msgr2=0x7f047c142d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c09f250 0x7f047c142d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f047400be50 tx=0x7f0474009f70 comp rx=0 tx=0).stop 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 -- 172.21.15.12:0/4137722625 shutdown_connections 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0468068580 0x7f046806aa40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f047c1463b0 0x7f047c0b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c09f250 0x7f047c142d90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 --2- 172.21.15.12:0/4137722625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c09acd0 0x7f047c0b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 -- 172.21.15.12:0/4137722625 >> 172.21.15.12:0/4137722625 conn(0x7f047c0b5e40 msgr2=0x7f047c13e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 -- 172.21.15.12:0/4137722625 shutdown_connections 2024-04-27T07:04:27.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f04830fb640 1 -- 172.21.15.12:0/4137722625 wait complete. 2024-04-27T07:04:27.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3cf7fff640 1 -- 172.21.15.12:0/1855938224 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d00056100 con 0x7f3d10157070 2024-04-27T07:04:27.995 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 -- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cfc068380 msgr2=0x7f3cfc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cfc068380 0x7f3cfc06a840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3d040b1000 tx=0x7f3d040aa070 comp rx=0 tx=0).stop 2024-04-27T07:04:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 -- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d10157070 msgr2=0x7f3d10089420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d10157070 0x7f3d10089420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000c780 tx=0x7f3d0000cc40 comp rx=0 tx=0).stop 2024-04-27T07:04:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 -- 172.21.15.12:0/1855938224 shutdown_connections 2024-04-27T07:04:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cfc068380 0x7f3cfc06a840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d10157070 0x7f3d10089420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3d1009ae50 0x7f3d100b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 --2- 172.21.15.12:0/1855938224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d1009aa70 0x7f3d101f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:27.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.989+0000 7f3d160e5640 1 -- 172.21.15.12:0/1855938224 >> 172.21.15.12:0/1855938224 conn(0x7f3d100b5e40 msgr2=0x7f3d100b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:27.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.993+0000 7f3d160e5640 1 -- 172.21.15.12:0/1855938224 shutdown_connections 2024-04-27T07:04:27.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.993+0000 7f3d160e5640 1 -- 172.21.15.12:0/1855938224 wait complete. 2024-04-27T07:04:28.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.997+0000 7fcd7adad640 1 -- 172.21.15.12:0/1879055241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcd58005590 con 0x7fcd740b8ec0 2024-04-27T07:04:28.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.997+0000 7f603d2f5640 1 -- 172.21.15.12:0/2332515903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f60300335d0 con 0x7f60300363a0 2024-04-27T07:04:28.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.997+0000 7fcd717fa640 1 -- 172.21.15.12:0/1879055241 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcd6c010040 con 0x7fcd740b8ec0 2024-04-27T07:04:28.002 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:28.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:27.997+0000 7f6034ff9640 1 -- 172.21.15.12:0/2332515903 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f602c0560e0 con 0x7f60300363a0 2024-04-27T07:04:28.003 INFO:teuthology.orchestra.run.smithi012.stdout:55834574856 2024-04-27T07:04:28.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 -- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcd640684c0 msgr2=0x7fcd6406a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcd640684c0 0x7fcd6406a980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd600b1000 tx=0x7fcd600a9000 comp rx=0 tx=0).stop 2024-04-27T07:04:28.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 -- 172.21.15.12:0/1879055241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740b8ec0 msgr2=0x7fcd741f1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740b8ec0 0x7fcd741f1150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00efc0 tx=0x7fcd6c009f90 comp rx=0 tx=0).stop 2024-04-27T07:04:28.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 -- 172.21.15.12:0/1879055241 shutdown_connections 2024-04-27T07:04:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcd640684c0 0x7fcd6406a980 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd740b8ec0 0x7fcd741f1150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcd740b73c0 0x7fcd74141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 --2- 172.21.15.12:0/1879055241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd740b5b00 0x7fcd7409b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 -- 172.21.15.12:0/1879055241 >> 172.21.15.12:0/1879055241 conn(0x7fcd740ba6d0 msgr2=0x7fcd74143820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 -- 172.21.15.12:0/1879055241 shutdown_connections 2024-04-27T07:04:28.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7fcd7adad640 1 -- 172.21.15.12:0/1879055241 wait complete. 2024-04-27T07:04:28.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 -- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f601c068450 msgr2=0x7f601c06a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f601c068450 0x7f601c06a910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60380656d0 tx=0x7f6038122070 comp rx=0 tx=0).stop 2024-04-27T07:04:28.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 -- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60300363a0 msgr2=0x7f60300c31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60300363a0 0x7f60300c31c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f602c00c430 tx=0x7f602c00c8f0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 -- 172.21.15.12:0/2332515903 shutdown_connections 2024-04-27T07:04:28.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f601c068450 0x7f601c06a910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60300c9050 0x7f6030033920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60300363a0 0x7f60300c31c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 --2- 172.21.15.12:0/2332515903 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6030004a40 0x7f6030169030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 -- 172.21.15.12:0/2332515903 >> 172.21.15.12:0/2332515903 conn(0x7f603003ace0 msgr2=0x7f6030004e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:28.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 -- 172.21.15.12:0/2332515903 shutdown_connections 2024-04-27T07:04:28.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.001+0000 7f603d2f5640 1 -- 172.21.15.12:0/2332515903 wait complete. 2024-04-27T07:04:28.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-04-27T07:04:28.017 DEBUG:teuthology.parallel:result is None 2024-04-27T07:04:28.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-04-27T07:04:28.024 DEBUG:teuthology.parallel:result is None 2024-04-27T07:04:28.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-04-27T07:04:28.025 DEBUG:teuthology.parallel:result is None 2024-04-27T07:04:28.034 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-04-27T07:04:28.034 DEBUG:teuthology.parallel:result is None 2024-04-27T07:04:28.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.037+0000 7f19b9463640 1 -- 172.21.15.12:0/2298245564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f19ac005720 con 0x7f19ac038e10 2024-04-27T07:04:28.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.037+0000 7f19b0ff9640 1 -- 172.21.15.12:0/2298245564 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f19a805b4c0 con 0x7f19ac038e10 2024-04-27T07:04:28.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.037+0000 7f0fde8a6640 1 -- 172.21.15.12:0/1911208134 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0fc0005860 con 0x7f0fd0037330 2024-04-27T07:04:28.043 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-04-27T07:04:28.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.037+0000 7f0fce7fc640 1 -- 172.21.15.12:0/1911208134 <== mon.2 v2:172.21.15.53:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0fd4056770 con 0x7f0fd0037330 2024-04-27T07:04:28.043 INFO:teuthology.orchestra.run.smithi012.stdout:55834574853 2024-04-27T07:04:28.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 -- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1998068380 msgr2=0x7f199806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1998068380 0x7f199806a840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009d50 tx=0x7f19a00090e0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 -- 172.21.15.12:0/2298245564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19ac038e10 msgr2=0x7f19ac0bea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19ac038e10 0x7f19ac0bea70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f19a800e930 tx=0x7f19a800edf0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 -- 172.21.15.12:0/2298245564 shutdown_connections 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1998068380 0x7f199806a840 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19ac0c40b0 0x7f19ac0c07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19ac038e10 0x7f19ac0bea70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 --2- 172.21.15.12:0/2298245564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f19ac037340 0x7f19ac0c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 -- 172.21.15.12:0/2298245564 >> 172.21.15.12:0/2298245564 conn(0x7f19ac03c6b0 msgr2=0x7f19ac03e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 -- 172.21.15.12:0/2298245564 shutdown_connections 2024-04-27T07:04:28.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f19b9463640 1 -- 172.21.15.12:0/2298245564 wait complete. 2024-04-27T07:04:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 -- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fbc068380 msgr2=0x7f0fbc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fbc068380 0x7f0fbc06a840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00c2a50 tx=0x7f0fc8009040 comp rx=0 tx=0).stop 2024-04-27T07:04:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 -- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fd0037330 msgr2=0x7f0fd00d9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fd0037330 0x7f0fd00d9d10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400c3a0 tx=0x7f0fd400c860 comp rx=0 tx=0).stop 2024-04-27T07:04:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 -- 172.21.15.12:0/1911208134 shutdown_connections 2024-04-27T07:04:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fbc068380 0x7f0fbc06a840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fd00b82c0 0x7f0fd0034160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fd00c95b0 0x7f0fd0168730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 --2- 172.21.15.12:0/1911208134 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fd0037330 0x7f0fd00d9d10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 -- 172.21.15.12:0/1911208134 >> 172.21.15.12:0/1911208134 conn(0x7f0fd003b120 msgr2=0x7f0fd0035e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:28.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 -- 172.21.15.12:0/1911208134 shutdown_connections 2024-04-27T07:04:28.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.041+0000 7f0fde8a6640 1 -- 172.21.15.12:0/1911208134 wait complete. 2024-04-27T07:04:28.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-04-27T07:04:28.062 DEBUG:teuthology.parallel:result is None 2024-04-27T07:04:28.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-04-27T07:04:28.062 DEBUG:teuthology.parallel:result is None 2024-04-27T07:04:28.063 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-27T07:04:28.063 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:04:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83baebf640 1 Processor -- start 2024-04-27T07:04:28.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83baebf640 1 -- start start 2024-04-27T07:04:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83baebf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f83b40b9930 0x7f83b40b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83baebf640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b41468c0 0x7f83b41490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83baebf640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b4149fc0 con 0x7f83b40b7830 2024-04-27T07:04:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83baebf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f83b4149ce0 con 0x7f83b40b9930 2024-04-27T07:04:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83baebf640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f83b4145c90 con 0x7f83b41468c0 2024-04-27T07:04:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b3fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f83b40b7830 0x7f83b41461b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59648/0 (socket says 172.21.15.12:59648) 2024-04-27T07:04:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b3fff640 1 -- 172.21.15.12:0/2710269362 learned_addr learned my addr 172.21.15.12:0/2710269362 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b8c5b640 1 --2- 172.21.15.12:0/2710269362 >> v2:172.21.15.53:3300/0 conn(0x7f83b40b9930 0x7f83b40b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b945c640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b41468c0 0x7f83b41490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219806735 0 0) 0x7f83b4149fc0 con 0x7f83b40b7830 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8398003530 con 0x7f83b40b7830 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3292851982 0 0) 0x7f8398003530 con 0x7f83b40b7830 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b41468c0 msgr2=0x7f83b41490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b41468c0 0x7f83b41490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 >> v2:172.21.15.53:3300/0 conn(0x7f83b40b9930 msgr2=0x7f83b40b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 --2- 172.21.15.12:0/2710269362 >> v2:172.21.15.53:3300/0 conn(0x7f83b40b9930 0x7f83b40b72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b40b9d10 con 0x7f83b40b7830 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f83a0001680 con 0x7f83b40b7830 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 >> v1:172.21.15.12:6789/0 conn(0x7f83b40b7830 legacy=0x7f83b41461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b41468c0 0x7f83980039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398003f40 0x7f8398004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8398004d50 0x7f83980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f83b4149ce0 con 0x7f8398003f40 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83b4145c90 con 0x7f8398004d50 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f839800ba20 con 0x7f83b41468c0 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b3fff640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398003f40 0x7f8398004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b8c5b640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b41468c0 0x7f83980039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b945c640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8398004d50 0x7f83980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b3fff640 1 -- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b41468c0 msgr2=0x7f83980039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b3fff640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b41468c0 0x7f83980039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b3fff640 1 -- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8398004d50 msgr2=0x7f83980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b3fff640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8398004d50 0x7f83980051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.209+0000 7f83b3fff640 1 -- 172.21.15.12:0/2710269362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8398003530 con 0x7f8398003f40 2024-04-27T07:04:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b3fff640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398003f40 0x7f8398004810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f83b40b7c50 tx=0x7f83a001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f3b3bcdb6f74ed server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:28.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0007d80 con 0x7f8398003f40 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a00a9020 con 0x7f8398003f40 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b37fe640 1 -- 172.21.15.12:0/2710269362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0005ae0 con 0x7f8398003f40 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 -- 172.21.15.12:0/2710269362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398003f40 msgr2=0x7f8398004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 --2- 172.21.15.12:0/2710269362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398003f40 0x7f8398004810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f83b40b7c50 tx=0x7f83a001cc60 comp rx=0 tx=0).stop 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b945c640 1 -- 172.21.15.12:0/2710269362 reap_dead start 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 -- 172.21.15.12:0/2710269362 shutdown_connections 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 -- 172.21.15.12:0/2710269362 >> 172.21.15.12:0/2710269362 conn(0x7f83b40ba740 msgr2=0x7f83b40bab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 -- 172.21.15.12:0/2710269362 shutdown_connections 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 -- 172.21.15.12:0/2710269362 wait complete. 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 Processor -- start 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 -- start start 2024-04-27T07:04:28.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8398003f40 0x7f83b409ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b41404f0 0x7f83b413f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b4140970 0x7f83b413f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f83b4150fe0 con 0x7f83b41404f0 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83b4150e60 con 0x7f8398003f40 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83baebf640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f83b4151160 con 0x7f83b4140970 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b41404f0 0x7f83b413f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b41404f0 0x7f83b413f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55624/0 (socket says 172.21.15.12:55624) 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b3fff640 1 -- 172.21.15.12:0/4290943990 learned_addr learned my addr 172.21.15.12:0/4290943990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b8c5b640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8398003f40 0x7f83b409ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b945c640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b4140970 0x7f83b413f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b3fff640 1 -- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b4140970 msgr2=0x7f83b413f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b3fff640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b4140970 0x7f83b413f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b3fff640 1 -- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8398003f40 msgr2=0x7f83b409ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b3fff640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8398003f40 0x7f83b409ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b3fff640 1 -- 172.21.15.12:0/4290943990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b4141d90 con 0x7f83b41404f0 2024-04-27T07:04:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b3fff640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b41404f0 0x7f83b413f350 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f83a00ab040 tx=0x7f8398005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b17fa640 1 -- 172.21.15.12:0/4290943990 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a00049f0 con 0x7f83b41404f0 2024-04-27T07:04:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b17fa640 1 -- 172.21.15.12:0/4290943990 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a0004b90 con 0x7f83b41404f0 2024-04-27T07:04:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.213+0000 7f83b17fa640 1 -- 172.21.15.12:0/4290943990 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0007d60 con 0x7f83b41404f0 2024-04-27T07:04:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.217+0000 7f83baebf640 1 -- 172.21.15.12:0/4290943990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b41418b0 con 0x7f83b41404f0 2024-04-27T07:04:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.217+0000 7f83baebf640 1 -- 172.21.15.12:0/4290943990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b409d100 con 0x7f83b41404f0 2024-04-27T07:04:28.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.217+0000 7f83baebf640 1 -- 172.21.15.12:0/4290943990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b409d300 con 0x7f83b41404f0 2024-04-27T07:04:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.217+0000 7f83b17fa640 1 -- 172.21.15.12:0/4290943990 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f83a0007070 con 0x7f83b41404f0 2024-04-27T07:04:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.217+0000 7f83b17fa640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f839c0683b0 0x7f839c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.217+0000 7f83b17fa640 1 -- 172.21.15.12:0/4290943990 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f83a0120220 con 0x7f83b41404f0 2024-04-27T07:04:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.217+0000 7f83b8c5b640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f839c0683b0 0x7f839c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.217+0000 7f83b8c5b640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f839c0683b0 0x7f839c06a870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83a4007520 tx=0x7f83a400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.221+0000 7f83b17fa640 1 -- 172.21.15.12:0/4290943990 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f83a0016970 con 0x7f83b41404f0 2024-04-27T07:04:28.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.377+0000 7f83baebf640 1 -- 172.21.15.12:0/4290943990 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83b4069330 con 0x7f839c0683b0 2024-04-27T07:04:28.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.393+0000 7f83b17fa640 1 -- 172.21.15.12:0/4290943990 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f83b4069330 con 0x7f839c0683b0 2024-04-27T07:04:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:28.402 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 -- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f839c0683b0 msgr2=0x7f839c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f839c0683b0 0x7f839c06a870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83a4007520 tx=0x7f83a400b040 comp rx=0 tx=0).stop 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 -- 172.21.15.12:0/4290943990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b41404f0 msgr2=0x7f83b413f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b41404f0 0x7f83b413f350 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f83a00ab040 tx=0x7f8398005840 comp rx=0 tx=0).stop 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 -- 172.21.15.12:0/4290943990 shutdown_connections 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f839c0683b0 0x7f839c06a870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83b4140970 0x7f83b413f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83b41404f0 0x7f83b413f350 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 --2- 172.21.15.12:0/4290943990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8398003f40 0x7f83b409ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 -- 172.21.15.12:0/4290943990 >> 172.21.15.12:0/4290943990 conn(0x7f83b40ba740 msgr2=0x7f83b4144d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 -- 172.21.15.12:0/4290943990 shutdown_connections 2024-04-27T07:04:28.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.397+0000 7f83baebf640 1 -- 172.21.15.12:0/4290943990 wait complete. 2024-04-27T07:04:28.418 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-27T07:04:26.611970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235636,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739084,"statfs":{"total":773094113280,"available":772852822016,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003684"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495786+0000","last_change":"2024-04-27T07:04:15.652893+0000","last_active":"2024-04-27T07:04:17.495786+0000","last_peered":"2024-04-27T07:04:17.495786+0000","last_clean":"2024-04-27T07:04:17.495786+0000","last_became_active":"2024-04-27T07:04:15.652155+0000","last_became_peered":"2024-04-27T07:04:15.652155+0000","last_unstale":"2024-04-27T07:04:17.495786+0000","last_undegraded":"2024-04-27T07:04:17.495786+0000","last_fullsized":"2024-04-27T07:04:17.495786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:07:07.313585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610741+0000","last_change":"2024-04-27T07:04:15.622035+0000","last_active":"2024-04-27T07:04:16.610741+0000","last_peered":"2024-04-27T07:04:16.610741+0000","last_clean":"2024-04-27T07:04:16.610741+0000","last_became_active":"2024-04-27T07:04:15.621365+0000","last_became_peered":"2024-04-27T07:04:15.621365+0000","last_unstale":"2024-04-27T07:04:16.610741+0000","last_undegraded":"2024-04-27T07:04:16.610741+0000","last_fullsized":"2024-04-27T07:04:16.610741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494175+0000","last_change":"2024-04-27T07:04:15.603108+0000","last_active":"2024-04-27T07:04:17.494175+0000","last_peered":"2024-04-27T07:04:17.494175+0000","last_clean":"2024-04-27T07:04:17.494175+0000","last_became_active":"2024-04-27T07:04:15.602672+0000","last_became_peered":"2024-04-27T07:04:15.602672+0000","last_unstale":"2024-04-27T07:04:17.494175+0000","last_undegraded":"2024-04-27T07:04:17.494175+0000","last_fullsized":"2024-04-27T07:04:17.494175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494169+0000","last_change":"2024-04-27T07:04:15.621096+0000","last_active":"2024-04-27T07:04:17.494169+0000","last_peered":"2024-04-27T07:04:17.494169+0000","last_clean":"2024-04-27T07:04:17.494169+0000","last_became_active":"2024-04-27T07:04:15.618832+0000","last_became_peered":"2024-04-27T07:04:15.618832+0000","last_unstale":"2024-04-27T07:04:17.494169+0000","last_undegraded":"2024-04-27T07:04:17.494169+0000","last_fullsized":"2024-04-27T07:04:17.494169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:08.890952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494305+0000","last_change":"2024-04-27T07:04:15.627875+0000","last_active":"2024-04-27T07:04:17.494305+0000","last_peered":"2024-04-27T07:04:17.494305+0000","last_clean":"2024-04-27T07:04:17.494305+0000","last_became_active":"2024-04-27T07:04:15.626904+0000","last_became_peered":"2024-04-27T07:04:15.626904+0000","last_unstale":"2024-04-27T07:04:17.494305+0000","last_undegraded":"2024-04-27T07:04:17.494305+0000","last_fullsized":"2024-04-27T07:04:17.494305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:11.386203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494240+0000","last_change":"2024-04-27T07:04:15.632294+0000","last_active":"2024-04-27T07:04:17.494240+0000","last_peered":"2024-04-27T07:04:17.494240+0000","last_clean":"2024-04-27T07:04:17.494240+0000","last_became_active":"2024-04-27T07:04:15.630191+0000","last_became_peered":"2024-04-27T07:04:15.630191+0000","last_unstale":"2024-04-27T07:04:17.494240+0000","last_undegraded":"2024-04-27T07:04:17.494240+0000","last_fullsized":"2024-04-27T07:04:17.494240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:19.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493321+0000","last_change":"2024-04-27T07:04:15.652899+0000","last_active":"2024-04-27T07:04:17.493321+0000","last_peered":"2024-04-27T07:04:17.493321+0000","last_clean":"2024-04-27T07:04:17.493321+0000","last_became_active":"2024-04-27T07:04:15.652383+0000","last_became_peered":"2024-04-27T07:04:15.652383+0000","last_unstale":"2024-04-27T07:04:17.493321+0000","last_undegraded":"2024-04-27T07:04:17.493321+0000","last_fullsized":"2024-04-27T07:04:17.493321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495664+0000","last_change":"2024-04-27T07:04:15.635192+0000","last_active":"2024-04-27T07:04:17.495664+0000","last_peered":"2024-04-27T07:04:17.495664+0000","last_clean":"2024-04-27T07:04:17.495664+0000","last_became_active":"2024-04-27T07:04:15.634845+0000","last_became_peered":"2024-04-27T07:04:15.634845+0000","last_unstale":"2024-04-27T07:04:17.495664+0000","last_undegraded":"2024-04-27T07:04:17.495664+0000","last_fullsized":"2024-04-27T07:04:17.495664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495704+0000","last_change":"2024-04-27T07:04:15.642203+0000","last_active":"2024-04-27T07:04:17.495704+0000","last_peered":"2024-04-27T07:04:17.495704+0000","last_clean":"2024-04-27T07:04:17.495704+0000","last_became_active":"2024-04-27T07:04:15.640450+0000","last_became_peered":"2024-04-27T07:04:15.640450+0000","last_unstale":"2024-04-27T07:04:17.495704+0000","last_undegraded":"2024-04-27T07:04:17.495704+0000","last_fullsized":"2024-04-27T07:04:17.495704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:46.491888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614056+0000","last_change":"2024-04-27T07:04:15.620338+0000","last_active":"2024-04-27T07:04:16.614056+0000","last_peered":"2024-04-27T07:04:16.614056+0000","last_clean":"2024-04-27T07:04:16.614056+0000","last_became_active":"2024-04-27T07:04:15.619521+0000","last_became_peered":"2024-04-27T07:04:15.619521+0000","last_unstale":"2024-04-27T07:04:16.614056+0000","last_undegraded":"2024-04-27T07:04:16.614056+0000","last_fullsized":"2024-04-27T07:04:16.614056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613051+0000","last_change":"2024-04-27T07:04:15.625615+0000","last_active":"2024-04-27T07:04:16.613051+0000","last_peered":"2024-04-27T07:04:16.613051+0000","last_clean":"2024-04-27T07:04:16.613051+0000","last_became_active":"2024-04-27T07:04:15.624734+0000","last_became_peered":"2024-04-27T07:04:15.624734+0000","last_unstale":"2024-04-27T07:04:16.613051+0000","last_undegraded":"2024-04-27T07:04:16.613051+0000","last_fullsized":"2024-04-27T07:04:16.613051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613805+0000","last_change":"2024-04-27T07:04:15.609501+0000","last_active":"2024-04-27T07:04:16.613805+0000","last_peered":"2024-04-27T07:04:16.613805+0000","last_clean":"2024-04-27T07:04:16.613805+0000","last_became_active":"2024-04-27T07:04:15.603709+0000","last_became_peered":"2024-04-27T07:04:15.603709+0000","last_unstale":"2024-04-27T07:04:16.613805+0000","last_undegraded":"2024-04-27T07:04:16.613805+0000","last_fullsized":"2024-04-27T07:04:16.613805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:59.982527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496346+0000","last_change":"2024-04-27T07:04:15.625180+0000","last_active":"2024-04-27T07:04:17.496346+0000","last_peered":"2024-04-27T07:04:17.496346+0000","last_clean":"2024-04-27T07:04:17.496346+0000","last_became_active":"2024-04-27T07:04:15.624748+0000","last_became_peered":"2024-04-27T07:04:15.624748+0000","last_unstale":"2024-04-27T07:04:17.496346+0000","last_undegraded":"2024-04-27T07:04:17.496346+0000","last_fullsized":"2024-04-27T07:04:17.496346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493065+0000","last_change":"2024-04-27T07:04:15.604939+0000","last_active":"2024-04-27T07:04:17.493065+0000","last_peered":"2024-04-27T07:04:17.493065+0000","last_clean":"2024-04-27T07:04:17.493065+0000","last_became_active":"2024-04-27T07:04:15.603580+0000","last_became_peered":"2024-04-27T07:04:15.603580+0000","last_unstale":"2024-04-27T07:04:17.493065+0000","last_undegraded":"2024-04-27T07:04:17.493065+0000","last_fullsized":"2024-04-27T07:04:17.493065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:45.914758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610468+0000","last_change":"2024-04-27T07:04:15.629060+0000","last_active":"2024-04-27T07:04:16.610468+0000","last_peered":"2024-04-27T07:04:16.610468+0000","last_clean":"2024-04-27T07:04:16.610468+0000","last_became_active":"2024-04-27T07:04:15.628737+0000","last_became_peered":"2024-04-27T07:04:15.628737+0000","last_unstale":"2024-04-27T07:04:16.610468+0000","last_undegraded":"2024-04-27T07:04:16.610468+0000","last_fullsized":"2024-04-27T07:04:16.610468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492723+0000","last_change":"2024-04-27T07:04:15.610164+0000","last_active":"2024-04-27T07:04:17.492723+0000","last_peered":"2024-04-27T07:04:17.492723+0000","last_clean":"2024-04-27T07:04:17.492723+0000","last_became_active":"2024-04-27T07:04:15.609290+0000","last_became_peered":"2024-04-27T07:04:15.609290+0000","last_unstale":"2024-04-27T07:04:17.492723+0000","last_undegraded":"2024-04-27T07:04:17.492723+0000","last_fullsized":"2024-04-27T07:04:17.492723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494381+0000","last_change":"2024-04-27T07:04:15.647816+0000","last_active":"2024-04-27T07:04:17.494381+0000","last_peered":"2024-04-27T07:04:17.494381+0000","last_clean":"2024-04-27T07:04:17.494381+0000","last_became_active":"2024-04-27T07:04:15.646892+0000","last_became_peered":"2024-04-27T07:04:15.646892+0000","last_unstale":"2024-04-27T07:04:17.494381+0000","last_undegraded":"2024-04-27T07:04:17.494381+0000","last_fullsized":"2024-04-27T07:04:17.494381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:01.732848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493299+0000","last_change":"2024-04-27T07:04:15.609407+0000","last_active":"2024-04-27T07:04:17.493299+0000","last_peered":"2024-04-27T07:04:17.493299+0000","last_clean":"2024-04-27T07:04:17.493299+0000","last_became_active":"2024-04-27T07:04:15.608565+0000","last_became_peered":"2024-04-27T07:04:15.608565+0000","last_unstale":"2024-04-27T07:04:17.493299+0000","last_undegraded":"2024-04-27T07:04:17.493299+0000","last_fullsized":"2024-04-27T07:04:17.493299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:37.376293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495719+0000","last_change":"2024-04-27T07:04:15.627082+0000","last_active":"2024-04-27T07:04:17.495719+0000","last_peered":"2024-04-27T07:04:17.495719+0000","last_clean":"2024-04-27T07:04:17.495719+0000","last_became_active":"2024-04-27T07:04:15.626302+0000","last_became_peered":"2024-04-27T07:04:15.626302+0000","last_unstale":"2024-04-27T07:04:17.495719+0000","last_undegraded":"2024-04-27T07:04:17.495719+0000","last_fullsized":"2024-04-27T07:04:17.495719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:01.732848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615116+0000","last_change":"2024-04-27T07:04:15.610661+0000","last_active":"2024-04-27T07:04:16.615116+0000","last_peered":"2024-04-27T07:04:16.615116+0000","last_clean":"2024-04-27T07:04:16.615116+0000","last_became_active":"2024-04-27T07:04:15.609899+0000","last_became_peered":"2024-04-27T07:04:15.609899+0000","last_unstale":"2024-04-27T07:04:16.615116+0000","last_undegraded":"2024-04-27T07:04:16.615116+0000","last_fullsized":"2024-04-27T07:04:16.615116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494321+0000","last_change":"2024-04-27T07:04:15.599714+0000","last_active":"2024-04-27T07:04:17.494321+0000","last_peered":"2024-04-27T07:04:17.494321+0000","last_clean":"2024-04-27T07:04:17.494321+0000","last_became_active":"2024-04-27T07:04:15.598855+0000","last_became_peered":"2024-04-27T07:04:15.598855+0000","last_unstale":"2024-04-27T07:04:17.494321+0000","last_undegraded":"2024-04-27T07:04:17.494321+0000","last_fullsized":"2024-04-27T07:04:17.494321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493334+0000","last_change":"2024-04-27T07:04:15.634881+0000","last_active":"2024-04-27T07:04:17.493334+0000","last_peered":"2024-04-27T07:04:17.493334+0000","last_clean":"2024-04-27T07:04:17.493334+0000","last_became_active":"2024-04-27T07:04:15.629428+0000","last_became_peered":"2024-04-27T07:04:15.629428+0000","last_unstale":"2024-04-27T07:04:17.493334+0000","last_undegraded":"2024-04-27T07:04:17.493334+0000","last_fullsized":"2024-04-27T07:04:17.493334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:40:20.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495828+0000","last_change":"2024-04-27T07:04:15.630015+0000","last_active":"2024-04-27T07:04:17.495828+0000","last_peered":"2024-04-27T07:04:17.495828+0000","last_clean":"2024-04-27T07:04:17.495828+0000","last_became_active":"2024-04-27T07:04:15.629177+0000","last_became_peered":"2024-04-27T07:04:15.629177+0000","last_unstale":"2024-04-27T07:04:17.495828+0000","last_undegraded":"2024-04-27T07:04:17.495828+0000","last_fullsized":"2024-04-27T07:04:17.495828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493333+0000","last_change":"2024-04-27T07:04:15.619427+0000","last_active":"2024-04-27T07:04:17.493333+0000","last_peered":"2024-04-27T07:04:17.493333+0000","last_clean":"2024-04-27T07:04:17.493333+0000","last_became_active":"2024-04-27T07:04:15.617987+0000","last_became_peered":"2024-04-27T07:04:15.617987+0000","last_unstale":"2024-04-27T07:04:17.493333+0000","last_undegraded":"2024-04-27T07:04:17.493333+0000","last_fullsized":"2024-04-27T07:04:17.493333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:39:04.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496704+0000","last_change":"2024-04-27T07:04:15.614547+0000","last_active":"2024-04-27T07:04:17.496704+0000","last_peered":"2024-04-27T07:04:17.496704+0000","last_clean":"2024-04-27T07:04:17.496704+0000","last_became_active":"2024-04-27T07:04:15.614210+0000","last_became_peered":"2024-04-27T07:04:15.614210+0000","last_unstale":"2024-04-27T07:04:17.496704+0000","last_undegraded":"2024-04-27T07:04:17.496704+0000","last_fullsized":"2024-04-27T07:04:17.496704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:40:20.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614461+0000","last_change":"2024-04-27T07:04:15.650072+0000","last_active":"2024-04-27T07:04:16.614461+0000","last_peered":"2024-04-27T07:04:16.614461+0000","last_clean":"2024-04-27T07:04:16.614461+0000","last_became_active":"2024-04-27T07:04:15.649461+0000","last_became_peered":"2024-04-27T07:04:15.649461+0000","last_unstale":"2024-04-27T07:04:16.614461+0000","last_undegraded":"2024-04-27T07:04:16.614461+0000","last_fullsized":"2024-04-27T07:04:16.614461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610542+0000","last_change":"2024-04-27T07:04:15.625742+0000","last_active":"2024-04-27T07:04:16.610542+0000","last_peered":"2024-04-27T07:04:16.610542+0000","last_clean":"2024-04-27T07:04:16.610542+0000","last_became_active":"2024-04-27T07:04:15.625049+0000","last_became_peered":"2024-04-27T07:04:15.625049+0000","last_unstale":"2024-04-27T07:04:16.610542+0000","last_undegraded":"2024-04-27T07:04:16.610542+0000","last_fullsized":"2024-04-27T07:04:16.610542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496813+0000","last_change":"2024-04-27T07:04:15.629376+0000","last_active":"2024-04-27T07:04:17.496813+0000","last_peered":"2024-04-27T07:04:17.496813+0000","last_clean":"2024-04-27T07:04:17.496813+0000","last_became_active":"2024-04-27T07:04:15.629058+0000","last_became_peered":"2024-04-27T07:04:15.629058+0000","last_unstale":"2024-04-27T07:04:17.496813+0000","last_undegraded":"2024-04-27T07:04:17.496813+0000","last_fullsized":"2024-04-27T07:04:17.496813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494539+0000","last_change":"2024-04-27T07:04:15.620883+0000","last_active":"2024-04-27T07:04:17.494539+0000","last_peered":"2024-04-27T07:04:17.494539+0000","last_clean":"2024-04-27T07:04:17.494539+0000","last_became_active":"2024-04-27T07:04:15.620549+0000","last_became_peered":"2024-04-27T07:04:15.620549+0000","last_unstale":"2024-04-27T07:04:17.494539+0000","last_undegraded":"2024-04-27T07:04:17.494539+0000","last_fullsized":"2024-04-27T07:04:17.494539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493778+0000","last_change":"2024-04-27T07:04:15.631001+0000","last_active":"2024-04-27T07:04:17.493778+0000","last_peered":"2024-04-27T07:04:17.493778+0000","last_clean":"2024-04-27T07:04:17.493778+0000","last_became_active":"2024-04-27T07:04:15.628448+0000","last_became_peered":"2024-04-27T07:04:15.628448+0000","last_unstale":"2024-04-27T07:04:17.493778+0000","last_undegraded":"2024-04-27T07:04:17.493778+0000","last_fullsized":"2024-04-27T07:04:17.493778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493518+0000","last_change":"2024-04-27T07:04:15.623687+0000","last_active":"2024-04-27T07:04:17.493518+0000","last_peered":"2024-04-27T07:04:17.493518+0000","last_clean":"2024-04-27T07:04:17.493518+0000","last_became_active":"2024-04-27T07:04:15.619092+0000","last_became_peered":"2024-04-27T07:04:15.619092+0000","last_unstale":"2024-04-27T07:04:17.493518+0000","last_undegraded":"2024-04-27T07:04:17.493518+0000","last_fullsized":"2024-04-27T07:04:17.493518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:50.960522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612331+0000","last_change":"2024-04-27T07:04:15.601950+0000","last_active":"2024-04-27T07:04:16.612331+0000","last_peered":"2024-04-27T07:04:16.612331+0000","last_clean":"2024-04-27T07:04:16.612331+0000","last_became_active":"2024-04-27T07:04:15.601144+0000","last_became_peered":"2024-04-27T07:04:15.601144+0000","last_unstale":"2024-04-27T07:04:16.612331+0000","last_undegraded":"2024-04-27T07:04:16.612331+0000","last_fullsized":"2024-04-27T07:04:16.612331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613553+0000","last_change":"2024-04-27T07:04:15.631943+0000","last_active":"2024-04-27T07:04:16.613553+0000","last_peered":"2024-04-27T07:04:16.613553+0000","last_clean":"2024-04-27T07:04:16.613553+0000","last_became_active":"2024-04-27T07:04:15.619207+0000","last_became_peered":"2024-04-27T07:04:15.619207+0000","last_unstale":"2024-04-27T07:04:16.613553+0000","last_undegraded":"2024-04-27T07:04:16.613553+0000","last_fullsized":"2024-04-27T07:04:16.613553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:58.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496502+0000","last_change":"2024-04-27T07:04:14.615664+0000","last_active":"2024-04-27T07:04:17.496502+0000","last_peered":"2024-04-27T07:04:17.496502+0000","last_clean":"2024-04-27T07:04:17.496502+0000","last_became_active":"2024-04-27T07:04:14.615220+0000","last_became_peered":"2024-04-27T07:04:14.615220+0000","last_unstale":"2024-04-27T07:04:17.496502+0000","last_undegraded":"2024-04-27T07:04:17.496502+0000","last_fullsized":"2024-04-27T07:04:17.496502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494747+0000","last_change":"2024-04-27T07:04:15.619876+0000","last_active":"2024-04-27T07:04:17.494747+0000","last_peered":"2024-04-27T07:04:17.494747+0000","last_clean":"2024-04-27T07:04:17.494747+0000","last_became_active":"2024-04-27T07:04:15.619131+0000","last_became_peered":"2024-04-27T07:04:15.619131+0000","last_unstale":"2024-04-27T07:04:17.494747+0000","last_undegraded":"2024-04-27T07:04:17.494747+0000","last_fullsized":"2024-04-27T07:04:17.494747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614112+0000","last_change":"2024-04-27T07:04:14.611792+0000","last_active":"2024-04-27T07:04:16.614112+0000","last_peered":"2024-04-27T07:04:16.614112+0000","last_clean":"2024-04-27T07:04:16.614112+0000","last_became_active":"2024-04-27T07:04:14.609221+0000","last_became_peered":"2024-04-27T07:04:14.609221+0000","last_unstale":"2024-04-27T07:04:16.614112+0000","last_undegraded":"2024-04-27T07:04:16.614112+0000","last_fullsized":"2024-04-27T07:04:16.614112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495969+0000","last_change":"2024-04-27T07:04:14.608168+0000","last_active":"2024-04-27T07:04:17.495969+0000","last_peered":"2024-04-27T07:04:17.495969+0000","last_clean":"2024-04-27T07:04:17.495969+0000","last_became_active":"2024-04-27T07:04:14.607735+0000","last_became_peered":"2024-04-27T07:04:14.607735+0000","last_unstale":"2024-04-27T07:04:17.495969+0000","last_undegraded":"2024-04-27T07:04:17.495969+0000","last_fullsized":"2024-04-27T07:04:17.495969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:06.395968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495767+0000","last_change":"2024-04-27T07:04:15.612016+0000","last_active":"2024-04-27T07:04:17.495767+0000","last_peered":"2024-04-27T07:04:17.495767+0000","last_clean":"2024-04-27T07:04:17.495767+0000","last_became_active":"2024-04-27T07:04:15.611658+0000","last_became_peered":"2024-04-27T07:04:15.611658+0000","last_unstale":"2024-04-27T07:04:17.495767+0000","last_undegraded":"2024-04-27T07:04:17.495767+0000","last_fullsized":"2024-04-27T07:04:17.495767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492906+0000","last_change":"2024-04-27T07:04:14.620476+0000","last_active":"2024-04-27T07:04:17.492906+0000","last_peered":"2024-04-27T07:04:17.492906+0000","last_clean":"2024-04-27T07:04:17.492906+0000","last_became_active":"2024-04-27T07:04:14.617893+0000","last_became_peered":"2024-04-27T07:04:14.617893+0000","last_unstale":"2024-04-27T07:04:17.492906+0000","last_undegraded":"2024-04-27T07:04:17.492906+0000","last_fullsized":"2024-04-27T07:04:17.492906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610222+0000","last_change":"2024-04-27T07:04:15.624591+0000","last_active":"2024-04-27T07:04:16.610222+0000","last_peered":"2024-04-27T07:04:16.610222+0000","last_clean":"2024-04-27T07:04:16.610222+0000","last_became_active":"2024-04-27T07:04:15.624299+0000","last_became_peered":"2024-04-27T07:04:15.624299+0000","last_unstale":"2024-04-27T07:04:16.610222+0000","last_undegraded":"2024-04-27T07:04:16.610222+0000","last_fullsized":"2024-04-27T07:04:16.610222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614771+0000","last_change":"2024-04-27T07:04:14.621378+0000","last_active":"2024-04-27T07:04:16.614771+0000","last_peered":"2024-04-27T07:04:16.614771+0000","last_clean":"2024-04-27T07:04:16.614771+0000","last_became_active":"2024-04-27T07:04:14.620678+0000","last_became_peered":"2024-04-27T07:04:14.620678+0000","last_unstale":"2024-04-27T07:04:16.614771+0000","last_undegraded":"2024-04-27T07:04:16.614771+0000","last_fullsized":"2024-04-27T07:04:16.614771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493724+0000","last_change":"2024-04-27T07:04:15.647396+0000","last_active":"2024-04-27T07:04:17.493724+0000","last_peered":"2024-04-27T07:04:17.493724+0000","last_clean":"2024-04-27T07:04:17.493724+0000","last_became_active":"2024-04-27T07:04:15.646774+0000","last_became_peered":"2024-04-27T07:04:15.646774+0000","last_unstale":"2024-04-27T07:04:17.493724+0000","last_undegraded":"2024-04-27T07:04:17.493724+0000","last_fullsized":"2024-04-27T07:04:17.493724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493229+0000","last_change":"2024-04-27T07:04:14.621935+0000","last_active":"2024-04-27T07:04:17.493229+0000","last_peered":"2024-04-27T07:04:17.493229+0000","last_clean":"2024-04-27T07:04:17.493229+0000","last_became_active":"2024-04-27T07:04:14.620120+0000","last_became_peered":"2024-04-27T07:04:14.620120+0000","last_unstale":"2024-04-27T07:04:17.493229+0000","last_undegraded":"2024-04-27T07:04:17.493229+0000","last_fullsized":"2024-04-27T07:04:17.493229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494443+0000","last_change":"2024-04-27T07:04:15.626451+0000","last_active":"2024-04-27T07:04:17.494443+0000","last_peered":"2024-04-27T07:04:17.494443+0000","last_clean":"2024-04-27T07:04:17.494443+0000","last_became_active":"2024-04-27T07:04:15.626147+0000","last_became_peered":"2024-04-27T07:04:15.626147+0000","last_unstale":"2024-04-27T07:04:17.494443+0000","last_undegraded":"2024-04-27T07:04:17.494443+0000","last_fullsized":"2024-04-27T07:04:17.494443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610433+0000","last_change":"2024-04-27T07:04:14.619618+0000","last_active":"2024-04-27T07:04:16.610433+0000","last_peered":"2024-04-27T07:04:16.610433+0000","last_clean":"2024-04-27T07:04:16.610433+0000","last_became_active":"2024-04-27T07:04:14.619155+0000","last_became_peered":"2024-04-27T07:04:14.619155+0000","last_unstale":"2024-04-27T07:04:16.610433+0000","last_undegraded":"2024-04-27T07:04:16.610433+0000","last_fullsized":"2024-04-27T07:04:16.610433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614234+0000","last_change":"2024-04-27T07:04:15.628583+0000","last_active":"2024-04-27T07:04:16.614234+0000","last_peered":"2024-04-27T07:04:16.614234+0000","last_clean":"2024-04-27T07:04:16.614234+0000","last_became_active":"2024-04-27T07:04:15.621801+0000","last_became_peered":"2024-04-27T07:04:15.621801+0000","last_unstale":"2024-04-27T07:04:16.614234+0000","last_undegraded":"2024-04-27T07:04:16.614234+0000","last_fullsized":"2024-04-27T07:04:16.614234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:37.376293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615199+0000","last_change":"2024-04-27T07:04:14.609138+0000","last_active":"2024-04-27T07:04:16.615199+0000","last_peered":"2024-04-27T07:04:16.615199+0000","last_clean":"2024-04-27T07:04:16.615199+0000","last_became_active":"2024-04-27T07:04:14.608374+0000","last_became_peered":"2024-04-27T07:04:14.608374+0000","last_unstale":"2024-04-27T07:04:16.615199+0000","last_undegraded":"2024-04-27T07:04:16.615199+0000","last_fullsized":"2024-04-27T07:04:16.615199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495184+0000","last_change":"2024-04-27T07:04:15.651834+0000","last_active":"2024-04-27T07:04:17.495184+0000","last_peered":"2024-04-27T07:04:17.495184+0000","last_clean":"2024-04-27T07:04:17.495184+0000","last_became_active":"2024-04-27T07:04:15.651286+0000","last_became_peered":"2024-04-27T07:04:15.651286+0000","last_unstale":"2024-04-27T07:04:17.495184+0000","last_undegraded":"2024-04-27T07:04:17.495184+0000","last_fullsized":"2024-04-27T07:04:17.495184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:00:38.115028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494797+0000","last_change":"2024-04-27T07:04:14.616280+0000","last_active":"2024-04-27T07:04:17.494797+0000","last_peered":"2024-04-27T07:04:17.494797+0000","last_clean":"2024-04-27T07:04:17.494797+0000","last_became_active":"2024-04-27T07:04:14.615920+0000","last_became_peered":"2024-04-27T07:04:14.615920+0000","last_unstale":"2024-04-27T07:04:17.494797+0000","last_undegraded":"2024-04-27T07:04:17.494797+0000","last_fullsized":"2024-04-27T07:04:17.494797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:06.395968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493937+0000","last_change":"2024-04-27T07:04:15.647042+0000","last_active":"2024-04-27T07:04:17.493937+0000","last_peered":"2024-04-27T07:04:17.493937+0000","last_clean":"2024-04-27T07:04:17.493937+0000","last_became_active":"2024-04-27T07:04:15.646683+0000","last_became_peered":"2024-04-27T07:04:15.646683+0000","last_unstale":"2024-04-27T07:04:17.493937+0000","last_undegraded":"2024-04-27T07:04:17.493937+0000","last_fullsized":"2024-04-27T07:04:17.493937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:05:55.398463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493914+0000","last_change":"2024-04-27T07:04:14.620284+0000","last_active":"2024-04-27T07:04:17.493914+0000","last_peered":"2024-04-27T07:04:17.493914+0000","last_clean":"2024-04-27T07:04:17.493914+0000","last_became_active":"2024-04-27T07:04:14.619379+0000","last_became_peered":"2024-04-27T07:04:14.619379+0000","last_unstale":"2024-04-27T07:04:17.493914+0000","last_undegraded":"2024-04-27T07:04:17.493914+0000","last_fullsized":"2024-04-27T07:04:17.493914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495014+0000","last_change":"2024-04-27T07:04:15.635161+0000","last_active":"2024-04-27T07:04:17.495014+0000","last_peered":"2024-04-27T07:04:17.495014+0000","last_clean":"2024-04-27T07:04:17.495014+0000","last_became_active":"2024-04-27T07:04:15.634423+0000","last_became_peered":"2024-04-27T07:04:15.634423+0000","last_unstale":"2024-04-27T07:04:17.495014+0000","last_undegraded":"2024-04-27T07:04:17.495014+0000","last_fullsized":"2024-04-27T07:04:17.495014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:17.226045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612693+0000","last_change":"2024-04-27T07:04:14.612975+0000","last_active":"2024-04-27T07:04:16.612693+0000","last_peered":"2024-04-27T07:04:16.612693+0000","last_clean":"2024-04-27T07:04:16.612693+0000","last_became_active":"2024-04-27T07:04:14.608937+0000","last_became_peered":"2024-04-27T07:04:14.608937+0000","last_unstale":"2024-04-27T07:04:16.612693+0000","last_undegraded":"2024-04-27T07:04:16.612693+0000","last_fullsized":"2024-04-27T07:04:16.612693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612667+0000","last_change":"2024-04-27T07:04:15.601024+0000","last_active":"2024-04-27T07:04:16.612667+0000","last_peered":"2024-04-27T07:04:16.612667+0000","last_clean":"2024-04-27T07:04:16.612667+0000","last_became_active":"2024-04-27T07:04:15.600245+0000","last_became_peered":"2024-04-27T07:04:15.600245+0000","last_unstale":"2024-04-27T07:04:16.612667+0000","last_undegraded":"2024-04-27T07:04:16.612667+0000","last_fullsized":"2024-04-27T07:04:16.612667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492667+0000","last_change":"2024-04-27T07:04:14.594793+0000","last_active":"2024-04-27T07:04:17.492667+0000","last_peered":"2024-04-27T07:04:17.492667+0000","last_clean":"2024-04-27T07:04:17.492667+0000","last_became_active":"2024-04-27T07:04:14.594022+0000","last_became_peered":"2024-04-27T07:04:14.594022+0000","last_unstale":"2024-04-27T07:04:17.492667+0000","last_undegraded":"2024-04-27T07:04:17.492667+0000","last_fullsized":"2024-04-27T07:04:17.492667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:01.729925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493727+0000","last_change":"2024-04-27T07:04:15.639932+0000","last_active":"2024-04-27T07:04:17.493727+0000","last_peered":"2024-04-27T07:04:17.493727+0000","last_clean":"2024-04-27T07:04:17.493727+0000","last_became_active":"2024-04-27T07:04:15.639192+0000","last_became_peered":"2024-04-27T07:04:15.639192+0000","last_unstale":"2024-04-27T07:04:17.493727+0000","last_undegraded":"2024-04-27T07:04:17.493727+0000","last_fullsized":"2024-04-27T07:04:17.493727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614105+0000","last_change":"2024-04-27T07:04:14.605930+0000","last_active":"2024-04-27T07:04:16.614105+0000","last_peered":"2024-04-27T07:04:16.614105+0000","last_clean":"2024-04-27T07:04:16.614105+0000","last_became_active":"2024-04-27T07:04:14.604957+0000","last_became_peered":"2024-04-27T07:04:14.604957+0000","last_unstale":"2024-04-27T07:04:16.614105+0000","last_undegraded":"2024-04-27T07:04:16.614105+0000","last_fullsized":"2024-04-27T07:04:16.614105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:15.465739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495558+0000","last_change":"2024-04-27T07:04:15.635261+0000","last_active":"2024-04-27T07:04:17.495558+0000","last_peered":"2024-04-27T07:04:17.495558+0000","last_clean":"2024-04-27T07:04:17.495558+0000","last_became_active":"2024-04-27T07:04:15.634020+0000","last_became_peered":"2024-04-27T07:04:15.634020+0000","last_unstale":"2024-04-27T07:04:17.495558+0000","last_undegraded":"2024-04-27T07:04:17.495558+0000","last_fullsized":"2024-04-27T07:04:17.495558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:08.890952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493458+0000","last_change":"2024-04-27T07:04:14.600680+0000","last_active":"2024-04-27T07:04:17.493458+0000","last_peered":"2024-04-27T07:04:17.493458+0000","last_clean":"2024-04-27T07:04:17.493458+0000","last_became_active":"2024-04-27T07:04:14.600233+0000","last_became_peered":"2024-04-27T07:04:14.600233+0000","last_unstale":"2024-04-27T07:04:17.493458+0000","last_undegraded":"2024-04-27T07:04:17.493458+0000","last_fullsized":"2024-04-27T07:04:17.493458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495918+0000","last_change":"2024-04-27T07:04:15.643279+0000","last_active":"2024-04-27T07:04:17.495918+0000","last_peered":"2024-04-27T07:04:17.495918+0000","last_clean":"2024-04-27T07:04:17.495918+0000","last_became_active":"2024-04-27T07:04:15.641303+0000","last_became_peered":"2024-04-27T07:04:15.641303+0000","last_unstale":"2024-04-27T07:04:17.495918+0000","last_undegraded":"2024-04-27T07:04:17.495918+0000","last_fullsized":"2024-04-27T07:04:17.495918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:02:46.002992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610599+0000","last_change":"2024-04-27T07:04:14.617808+0000","last_active":"2024-04-27T07:04:16.610599+0000","last_peered":"2024-04-27T07:04:16.610599+0000","last_clean":"2024-04-27T07:04:16.610599+0000","last_became_active":"2024-04-27T07:04:14.617505+0000","last_became_peered":"2024-04-27T07:04:14.617505+0000","last_unstale":"2024-04-27T07:04:16.610599+0000","last_undegraded":"2024-04-27T07:04:16.610599+0000","last_fullsized":"2024-04-27T07:04:16.610599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494318+0000","last_change":"2024-04-27T07:04:15.619586+0000","last_active":"2024-04-27T07:04:17.494318+0000","last_peered":"2024-04-27T07:04:17.494318+0000","last_clean":"2024-04-27T07:04:17.494318+0000","last_became_active":"2024-04-27T07:04:15.617842+0000","last_became_peered":"2024-04-27T07:04:15.617842+0000","last_unstale":"2024-04-27T07:04:17.494318+0000","last_undegraded":"2024-04-27T07:04:17.494318+0000","last_fullsized":"2024-04-27T07:04:17.494318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:46.491888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493601+0000","last_change":"2024-04-27T07:04:14.619149+0000","last_active":"2024-04-27T07:04:17.493601+0000","last_peered":"2024-04-27T07:04:17.493601+0000","last_clean":"2024-04-27T07:04:17.493601+0000","last_became_active":"2024-04-27T07:04:14.618660+0000","last_became_peered":"2024-04-27T07:04:14.618660+0000","last_unstale":"2024-04-27T07:04:17.493601+0000","last_undegraded":"2024-04-27T07:04:17.493601+0000","last_fullsized":"2024-04-27T07:04:17.493601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495600+0000","last_change":"2024-04-27T07:04:15.640980+0000","last_active":"2024-04-27T07:04:17.495600+0000","last_peered":"2024-04-27T07:04:17.495600+0000","last_clean":"2024-04-27T07:04:17.495600+0000","last_became_active":"2024-04-27T07:04:15.640289+0000","last_became_peered":"2024-04-27T07:04:15.640289+0000","last_unstale":"2024-04-27T07:04:17.495600+0000","last_undegraded":"2024-04-27T07:04:17.495600+0000","last_fullsized":"2024-04-27T07:04:17.495600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:19.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492606+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:04:17.492606+0000","last_peered":"2024-04-27T07:04:17.492606+0000","last_clean":"2024-04-27T07:04:17.492606+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:04:17.492606+0000","last_undegraded":"2024-04-27T07:04:17.492606+0000","last_fullsized":"2024-04-27T07:04:17.492606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493168+0000","last_change":"2024-04-27T07:04:14.621914+0000","last_active":"2024-04-27T07:04:17.493168+0000","last_peered":"2024-04-27T07:04:17.493168+0000","last_clean":"2024-04-27T07:04:17.493168+0000","last_became_active":"2024-04-27T07:04:14.620901+0000","last_became_peered":"2024-04-27T07:04:14.620901+0000","last_unstale":"2024-04-27T07:04:17.493168+0000","last_undegraded":"2024-04-27T07:04:17.493168+0000","last_fullsized":"2024-04-27T07:04:17.493168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496303+0000","last_change":"2024-04-27T07:04:15.635276+0000","last_active":"2024-04-27T07:04:17.496303+0000","last_peered":"2024-04-27T07:04:17.496303+0000","last_clean":"2024-04-27T07:04:17.496303+0000","last_became_active":"2024-04-27T07:04:15.634946+0000","last_became_peered":"2024-04-27T07:04:15.634946+0000","last_unstale":"2024-04-27T07:04:17.496303+0000","last_undegraded":"2024-04-27T07:04:17.496303+0000","last_fullsized":"2024-04-27T07:04:17.496303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492322+0000","last_change":"2024-04-27T07:04:14.628608+0000","last_active":"2024-04-27T07:04:17.492322+0000","last_peered":"2024-04-27T07:04:17.492322+0000","last_clean":"2024-04-27T07:04:17.492322+0000","last_became_active":"2024-04-27T07:04:14.626795+0000","last_became_peered":"2024-04-27T07:04:14.626795+0000","last_unstale":"2024-04-27T07:04:17.492322+0000","last_undegraded":"2024-04-27T07:04:17.492322+0000","last_fullsized":"2024-04-27T07:04:17.492322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:32:26.049320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493125+0000","last_change":"2024-04-27T07:04:15.604705+0000","last_active":"2024-04-27T07:04:17.493125+0000","last_peered":"2024-04-27T07:04:17.493125+0000","last_clean":"2024-04-27T07:04:17.493125+0000","last_became_active":"2024-04-27T07:04:15.603226+0000","last_became_peered":"2024-04-27T07:04:15.603226+0000","last_unstale":"2024-04-27T07:04:17.493125+0000","last_undegraded":"2024-04-27T07:04:17.493125+0000","last_fullsized":"2024-04-27T07:04:17.493125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:24.246666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492975+0000","last_change":"2024-04-27T07:04:14.601261+0000","last_active":"2024-04-27T07:04:17.492975+0000","last_peered":"2024-04-27T07:04:17.492975+0000","last_clean":"2024-04-27T07:04:17.492975+0000","last_became_active":"2024-04-27T07:04:14.600555+0000","last_became_peered":"2024-04-27T07:04:14.600555+0000","last_unstale":"2024-04-27T07:04:17.492975+0000","last_undegraded":"2024-04-27T07:04:17.492975+0000","last_fullsized":"2024-04-27T07:04:17.492975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610538+0000","last_change":"2024-04-27T07:04:15.632176+0000","last_active":"2024-04-27T07:04:16.610538+0000","last_peered":"2024-04-27T07:04:16.610538+0000","last_clean":"2024-04-27T07:04:16.610538+0000","last_became_active":"2024-04-27T07:04:15.631861+0000","last_became_peered":"2024-04-27T07:04:15.631861+0000","last_unstale":"2024-04-27T07:04:16.610538+0000","last_undegraded":"2024-04-27T07:04:16.610538+0000","last_fullsized":"2024-04-27T07:04:16.610538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494374+0000","last_change":"2024-04-27T07:04:14.623584+0000","last_active":"2024-04-27T07:04:17.494374+0000","last_peered":"2024-04-27T07:04:17.494374+0000","last_clean":"2024-04-27T07:04:17.494374+0000","last_became_active":"2024-04-27T07:04:14.620709+0000","last_became_peered":"2024-04-27T07:04:14.620709+0000","last_unstale":"2024-04-27T07:04:17.494374+0000","last_undegraded":"2024-04-27T07:04:17.494374+0000","last_fullsized":"2024-04-27T07:04:17.494374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614757+0000","last_change":"2024-04-27T07:04:15.633827+0000","last_active":"2024-04-27T07:04:16.614757+0000","last_peered":"2024-04-27T07:04:16.614757+0000","last_clean":"2024-04-27T07:04:16.614757+0000","last_became_active":"2024-04-27T07:04:15.632912+0000","last_became_peered":"2024-04-27T07:04:15.632912+0000","last_unstale":"2024-04-27T07:04:16.614757+0000","last_undegraded":"2024-04-27T07:04:16.614757+0000","last_fullsized":"2024-04-27T07:04:16.614757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:50.960522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610382+0000","last_change":"2024-04-27T07:04:14.603584+0000","last_active":"2024-04-27T07:04:16.610382+0000","last_peered":"2024-04-27T07:04:16.610382+0000","last_clean":"2024-04-27T07:04:16.610382+0000","last_became_active":"2024-04-27T07:04:14.602186+0000","last_became_peered":"2024-04-27T07:04:14.602186+0000","last_unstale":"2024-04-27T07:04:16.610382+0000","last_undegraded":"2024-04-27T07:04:16.610382+0000","last_fullsized":"2024-04-27T07:04:16.610382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494048+0000","last_change":"2024-04-27T07:04:15.620627+0000","last_active":"2024-04-27T07:04:17.494048+0000","last_peered":"2024-04-27T07:04:17.494048+0000","last_clean":"2024-04-27T07:04:17.494048+0000","last_became_active":"2024-04-27T07:04:15.619725+0000","last_became_peered":"2024-04-27T07:04:15.619725+0000","last_unstale":"2024-04-27T07:04:17.494048+0000","last_undegraded":"2024-04-27T07:04:17.494048+0000","last_fullsized":"2024-04-27T07:04:17.494048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:17.226045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495737+0000","last_change":"2024-04-27T07:04:14.630298+0000","last_active":"2024-04-27T07:04:17.495737+0000","last_peered":"2024-04-27T07:04:17.495737+0000","last_clean":"2024-04-27T07:04:17.495737+0000","last_became_active":"2024-04-27T07:04:14.629539+0000","last_became_peered":"2024-04-27T07:04:14.629539+0000","last_unstale":"2024-04-27T07:04:17.495737+0000","last_undegraded":"2024-04-27T07:04:17.495737+0000","last_fullsized":"2024-04-27T07:04:17.495737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:44.902463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610300+0000","last_change":"2024-04-27T07:04:15.604054+0000","last_active":"2024-04-27T07:04:16.610300+0000","last_peered":"2024-04-27T07:04:16.610300+0000","last_clean":"2024-04-27T07:04:16.610300+0000","last_became_active":"2024-04-27T07:04:15.603243+0000","last_became_peered":"2024-04-27T07:04:15.603243+0000","last_unstale":"2024-04-27T07:04:16.610300+0000","last_undegraded":"2024-04-27T07:04:16.610300+0000","last_fullsized":"2024-04-27T07:04:16.610300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610793+0000","last_change":"2024-04-27T07:04:14.614799+0000","last_active":"2024-04-27T07:04:16.610793+0000","last_peered":"2024-04-27T07:04:16.610793+0000","last_clean":"2024-04-27T07:04:16.610793+0000","last_became_active":"2024-04-27T07:04:14.614348+0000","last_became_peered":"2024-04-27T07:04:14.614348+0000","last_unstale":"2024-04-27T07:04:16.610793+0000","last_undegraded":"2024-04-27T07:04:16.610793+0000","last_fullsized":"2024-04-27T07:04:16.610793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:49.948227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.497067+0000","last_change":"2024-04-27T07:04:15.649374+0000","last_active":"2024-04-27T07:04:17.497067+0000","last_peered":"2024-04-27T07:04:17.497067+0000","last_clean":"2024-04-27T07:04:17.497067+0000","last_became_active":"2024-04-27T07:04:15.649079+0000","last_became_peered":"2024-04-27T07:04:15.649079+0000","last_unstale":"2024-04-27T07:04:17.497067+0000","last_undegraded":"2024-04-27T07:04:17.497067+0000","last_fullsized":"2024-04-27T07:04:17.497067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615045+0000","last_change":"2024-04-27T07:04:14.630929+0000","last_active":"2024-04-27T07:04:16.615045+0000","last_peered":"2024-04-27T07:04:16.615045+0000","last_clean":"2024-04-27T07:04:16.615045+0000","last_became_active":"2024-04-27T07:04:14.630345+0000","last_became_peered":"2024-04-27T07:04:14.630345+0000","last_unstale":"2024-04-27T07:04:16.615045+0000","last_undegraded":"2024-04-27T07:04:16.615045+0000","last_fullsized":"2024-04-27T07:04:16.615045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494516+0000","last_change":"2024-04-27T07:04:15.605948+0000","last_active":"2024-04-27T07:04:17.494516+0000","last_peered":"2024-04-27T07:04:17.494516+0000","last_clean":"2024-04-27T07:04:17.494516+0000","last_became_active":"2024-04-27T07:04:15.603763+0000","last_became_peered":"2024-04-27T07:04:15.603763+0000","last_unstale":"2024-04-27T07:04:17.494516+0000","last_undegraded":"2024-04-27T07:04:17.494516+0000","last_fullsized":"2024-04-27T07:04:17.494516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492302+0000","last_change":"2024-04-27T07:04:14.628661+0000","last_active":"2024-04-27T07:04:17.492302+0000","last_peered":"2024-04-27T07:04:17.492302+0000","last_clean":"2024-04-27T07:04:17.492302+0000","last_became_active":"2024-04-27T07:04:14.626731+0000","last_became_peered":"2024-04-27T07:04:14.626731+0000","last_unstale":"2024-04-27T07:04:17.492302+0000","last_undegraded":"2024-04-27T07:04:17.492302+0000","last_fullsized":"2024-04-27T07:04:17.492302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613464+0000","last_change":"2024-04-27T07:04:15.608033+0000","last_active":"2024-04-27T07:04:16.613464+0000","last_peered":"2024-04-27T07:04:16.613464+0000","last_clean":"2024-04-27T07:04:16.613464+0000","last_became_active":"2024-04-27T07:04:15.607219+0000","last_became_peered":"2024-04-27T07:04:15.607219+0000","last_unstale":"2024-04-27T07:04:16.613464+0000","last_undegraded":"2024-04-27T07:04:16.613464+0000","last_fullsized":"2024-04-27T07:04:16.613464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:02.140237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610281+0000","last_change":"2024-04-27T07:04:14.613406+0000","last_active":"2024-04-27T07:04:16.610281+0000","last_peered":"2024-04-27T07:04:16.610281+0000","last_clean":"2024-04-27T07:04:16.610281+0000","last_became_active":"2024-04-27T07:04:14.613017+0000","last_became_peered":"2024-04-27T07:04:14.613017+0000","last_unstale":"2024-04-27T07:04:16.610281+0000","last_undegraded":"2024-04-27T07:04:16.610281+0000","last_fullsized":"2024-04-27T07:04:16.610281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496138+0000","last_change":"2024-04-27T07:04:15.617597+0000","last_active":"2024-04-27T07:04:17.496138+0000","last_peered":"2024-04-27T07:04:17.496138+0000","last_clean":"2024-04-27T07:04:17.496138+0000","last_became_active":"2024-04-27T07:04:15.616122+0000","last_became_peered":"2024-04-27T07:04:15.616122+0000","last_unstale":"2024-04-27T07:04:17.496138+0000","last_undegraded":"2024-04-27T07:04:17.496138+0000","last_fullsized":"2024-04-27T07:04:17.496138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610542+0000","last_change":"2024-04-27T07:04:14.604022+0000","last_active":"2024-04-27T07:04:16.610542+0000","last_peered":"2024-04-27T07:04:16.610542+0000","last_clean":"2024-04-27T07:04:16.610542+0000","last_became_active":"2024-04-27T07:04:14.603308+0000","last_became_peered":"2024-04-27T07:04:14.603308+0000","last_unstale":"2024-04-27T07:04:16.610542+0000","last_undegraded":"2024-04-27T07:04:16.610542+0000","last_fullsized":"2024-04-27T07:04:16.610542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496561+0000","last_change":"2024-04-27T07:04:15.620378+0000","last_active":"2024-04-27T07:04:17.496561+0000","last_peered":"2024-04-27T07:04:17.496561+0000","last_clean":"2024-04-27T07:04:17.496561+0000","last_became_active":"2024-04-27T07:04:15.620100+0000","last_became_peered":"2024-04-27T07:04:15.620100+0000","last_unstale":"2024-04-27T07:04:17.496561+0000","last_undegraded":"2024-04-27T07:04:17.496561+0000","last_fullsized":"2024-04-27T07:04:17.496561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494886+0000","last_change":"2024-04-27T07:04:14.629851+0000","last_active":"2024-04-27T07:04:17.494886+0000","last_peered":"2024-04-27T07:04:17.494886+0000","last_clean":"2024-04-27T07:04:17.494886+0000","last_became_active":"2024-04-27T07:04:14.629063+0000","last_became_peered":"2024-04-27T07:04:14.629063+0000","last_unstale":"2024-04-27T07:04:17.494886+0000","last_undegraded":"2024-04-27T07:04:17.494886+0000","last_fullsized":"2024-04-27T07:04:17.494886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:49.948227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613962+0000","last_change":"2024-04-27T07:04:15.616289+0000","last_active":"2024-04-27T07:04:16.613962+0000","last_peered":"2024-04-27T07:04:16.613962+0000","last_clean":"2024-04-27T07:04:16.613962+0000","last_became_active":"2024-04-27T07:04:15.615386+0000","last_became_peered":"2024-04-27T07:04:15.615386+0000","last_unstale":"2024-04-27T07:04:16.613962+0000","last_undegraded":"2024-04-27T07:04:16.613962+0000","last_fullsized":"2024-04-27T07:04:16.613962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493841+0000","last_change":"2024-04-27T07:04:15.646050+0000","last_active":"2024-04-27T07:04:17.493841+0000","last_peered":"2024-04-27T07:04:17.493841+0000","last_clean":"2024-04-27T07:04:17.493841+0000","last_became_active":"2024-04-27T07:04:15.645284+0000","last_became_peered":"2024-04-27T07:04:15.645284+0000","last_unstale":"2024-04-27T07:04:17.493841+0000","last_undegraded":"2024-04-27T07:04:17.493841+0000","last_fullsized":"2024-04-27T07:04:17.493841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:11.386203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494836+0000","last_change":"2024-04-27T07:04:14.629860+0000","last_active":"2024-04-27T07:04:17.494836+0000","last_peered":"2024-04-27T07:04:17.494836+0000","last_clean":"2024-04-27T07:04:17.494836+0000","last_became_active":"2024-04-27T07:04:14.629138+0000","last_became_peered":"2024-04-27T07:04:14.629138+0000","last_unstale":"2024-04-27T07:04:17.494836+0000","last_undegraded":"2024-04-27T07:04:17.494836+0000","last_fullsized":"2024-04-27T07:04:17.494836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494061+0000","last_change":"2024-04-27T07:04:15.638833+0000","last_active":"2024-04-27T07:04:17.494061+0000","last_peered":"2024-04-27T07:04:17.494061+0000","last_clean":"2024-04-27T07:04:17.494061+0000","last_became_active":"2024-04-27T07:04:15.638192+0000","last_became_peered":"2024-04-27T07:04:15.638192+0000","last_unstale":"2024-04-27T07:04:17.494061+0000","last_undegraded":"2024-04-27T07:04:17.494061+0000","last_fullsized":"2024-04-27T07:04:17.494061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:00:38.115028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495394+0000","last_change":"2024-04-27T07:04:14.632528+0000","last_active":"2024-04-27T07:04:17.495394+0000","last_peered":"2024-04-27T07:04:17.495394+0000","last_clean":"2024-04-27T07:04:17.495394+0000","last_became_active":"2024-04-27T07:04:14.631089+0000","last_became_peered":"2024-04-27T07:04:14.631089+0000","last_unstale":"2024-04-27T07:04:17.495394+0000","last_undegraded":"2024-04-27T07:04:17.495394+0000","last_fullsized":"2024-04-27T07:04:17.495394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:03:50.734904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494459+0000","last_change":"2024-04-27T07:04:15.626754+0000","last_active":"2024-04-27T07:04:17.494459+0000","last_peered":"2024-04-27T07:04:17.494459+0000","last_clean":"2024-04-27T07:04:17.494459+0000","last_became_active":"2024-04-27T07:04:15.626499+0000","last_became_peered":"2024-04-27T07:04:15.626499+0000","last_unstale":"2024-04-27T07:04:17.494459+0000","last_undegraded":"2024-04-27T07:04:17.494459+0000","last_fullsized":"2024-04-27T07:04:17.494459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496732+0000","last_change":"2024-04-27T07:04:14.598918+0000","last_active":"2024-04-27T07:04:17.496732+0000","last_peered":"2024-04-27T07:04:17.496732+0000","last_clean":"2024-04-27T07:04:17.496732+0000","last_became_active":"2024-04-27T07:04:14.598504+0000","last_became_peered":"2024-04-27T07:04:14.598504+0000","last_unstale":"2024-04-27T07:04:17.496732+0000","last_undegraded":"2024-04-27T07:04:17.496732+0000","last_fullsized":"2024-04-27T07:04:17.496732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:01.729925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496574+0000","last_change":"2024-04-27T07:04:15.625377+0000","last_active":"2024-04-27T07:04:17.496574+0000","last_peered":"2024-04-27T07:04:17.496574+0000","last_clean":"2024-04-27T07:04:17.496574+0000","last_became_active":"2024-04-27T07:04:15.625075+0000","last_became_peered":"2024-04-27T07:04:15.625075+0000","last_unstale":"2024-04-27T07:04:17.496574+0000","last_undegraded":"2024-04-27T07:04:17.496574+0000","last_fullsized":"2024-04-27T07:04:17.496574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:39:04.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494529+0000","last_change":"2024-04-27T07:04:14.626854+0000","last_active":"2024-04-27T07:04:17.494529+0000","last_peered":"2024-04-27T07:04:17.494529+0000","last_clean":"2024-04-27T07:04:17.494529+0000","last_became_active":"2024-04-27T07:04:14.626588+0000","last_became_peered":"2024-04-27T07:04:14.626588+0000","last_unstale":"2024-04-27T07:04:17.494529+0000","last_undegraded":"2024-04-27T07:04:17.494529+0000","last_fullsized":"2024-04-27T07:04:17.494529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:21.930932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:04:28.422 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:04:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.573+0000 7f26f3530640 1 Processor -- start 2024-04-27T07:04:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.573+0000 7f26f3530640 1 -- start start 2024-04-27T07:04:28.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f26ec0b64c0 0x7f26ec0b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26ec0b7560 con 0x7f26ec0b5ae0 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26ec0b68a0 con 0x7f26ec148810 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f26ec148c80 con 0x7f26ec0b64c0 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f12cc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f26ec0b64c0 0x7f26ec0b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f0acb640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f26ec0b5ae0 0x7f26ec141ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59650/0 (socket says 172.21.15.12:59650) 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f0acb640 1 -- 172.21.15.12:0/626070775 learned_addr learned my addr 172.21.15.12:0/626070775 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26e3fff640 1 -- 172.21.15.12:0/626070775 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 935453580 0 0) 0x7f26ec0b7560 con 0x7f26ec0b5ae0 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26e3fff640 1 -- 172.21.15.12:0/626070775 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26d8003530 con 0x7f26ec0b5ae0 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f12cc640 1 -- 172.21.15.12:0/626070775 >> v1:172.21.15.53:6789/0 conn(0x7f26ec148810 legacy=0x7f26ec152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f12cc640 1 -- 172.21.15.12:0/626070775 >> v1:172.21.15.12:6789/0 conn(0x7f26ec0b5ae0 legacy=0x7f26ec141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f12cc640 1 -- 172.21.15.12:0/626070775 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec148010 con 0x7f26ec0b64c0 2024-04-27T07:04:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f12cc640 1 --2- 172.21.15.12:0/626070775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 0x7f26ec0b7f60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009d90 tx=0x7f26e801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32df3688ed6e9615 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26e3fff640 1 -- 172.21.15.12:0/626070775 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80b2070 con 0x7f26ec0b64c0 2024-04-27T07:04:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26e3fff640 1 -- 172.21.15.12:0/626070775 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e80a5820 con 0x7f26ec0b64c0 2024-04-27T07:04:28.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26e3fff640 1 -- 172.21.15.12:0/626070775 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80a5bf0 con 0x7f26ec0b64c0 2024-04-27T07:04:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 -- 172.21.15.12:0/626070775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 msgr2=0x7f26ec0b7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 --2- 172.21.15.12:0/626070775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 0x7f26ec0b7f60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009d90 tx=0x7f26e801c850 comp rx=0 tx=0).stop 2024-04-27T07:04:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 -- 172.21.15.12:0/626070775 shutdown_connections 2024-04-27T07:04:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 --2- 172.21.15.12:0/626070775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 0x7f26ec0b7f60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 -- 172.21.15.12:0/626070775 >> 172.21.15.12:0/626070775 conn(0x7f26ec073c00 msgr2=0x7f26ec0bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 -- 172.21.15.12:0/626070775 shutdown_connections 2024-04-27T07:04:28.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.577+0000 7f26f3530640 1 -- 172.21.15.12:0/626070775 wait complete. 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 Processor -- start 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 -- start start 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26ec0b5ae0 0x7f26ec13a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 0x7f26ec1e3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f26ec148810 0x7f26ec135a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec142360 con 0x7f26ec0b64c0 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec1421e0 con 0x7f26ec0b5ae0 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f26ec156580 con 0x7f26ec148810 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f0acb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 0x7f26ec1e3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f0acb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 0x7f26ec1e3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55644/0 (socket says 172.21.15.12:55644) 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f0acb640 1 -- 172.21.15.12:0/334387480 learned_addr learned my addr 172.21.15.12:0/334387480 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:28.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f0acb640 1 -- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f26ec148810 msgr2=0x7f26ec135a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f1acd640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f26ec148810 0x7f26ec135a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f12cc640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26ec0b5ae0 0x7f26ec13a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f0acb640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f26ec148810 0x7f26ec135a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f0acb640 1 -- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26ec0b5ae0 msgr2=0x7f26ec13a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f0acb640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26ec0b5ae0 0x7f26ec13a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f0acb640 1 -- 172.21.15.12:0/334387480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec13e640 con 0x7f26ec0b64c0 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f0acb640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 0x7f26ec1e3500 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00c450 tx=0x7f26dc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26e1ffb640 1 -- 172.21.15.12:0/334387480 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc017070 con 0x7f26ec0b64c0 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26e1ffb640 1 -- 172.21.15.12:0/334387480 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26dc005cf0 con 0x7f26ec0b64c0 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 -- 172.21.15.12:0/334387480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ec13e920 con 0x7f26ec0b64c0 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26e1ffb640 1 -- 172.21.15.12:0/334387480 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc007270 con 0x7f26ec0b64c0 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 -- 172.21.15.12:0/334387480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ec13f480 con 0x7f26ec0b64c0 2024-04-27T07:04:28.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f3530640 1 -- 172.21.15.12:0/334387480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ec137090 con 0x7f26ec0b64c0 2024-04-27T07:04:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26e1ffb640 1 -- 172.21.15.12:0/334387480 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f26dc01a070 con 0x7f26ec0b64c0 2024-04-27T07:04:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26e1ffb640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f26d00683b0 0x7f26d006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26e1ffb640 1 -- 172.21.15.12:0/334387480 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f26dc085c30 con 0x7f26ec0b64c0 2024-04-27T07:04:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.581+0000 7f26f12cc640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f26d00683b0 0x7f26d006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.585+0000 7f26f12cc640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f26d00683b0 0x7f26d006a870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f26e80af000 tx=0x7f26e80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:28.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.589+0000 7f26e1ffb640 1 -- 172.21.15.12:0/334387480 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26dc056340 con 0x7f26ec0b64c0 2024-04-27T07:04:28.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.741+0000 7f26f3530640 1 -- 172.21.15.12:0/334387480 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26ec0b53e0 con 0x7f26d00683b0 2024-04-27T07:04:28.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.761+0000 7f26e1ffb640 1 -- 172.21.15.12:0/334387480 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f26ec0b53e0 con 0x7f26d00683b0 2024-04-27T07:04:28.767 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:28.769 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:04:28.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 -- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f26d00683b0 msgr2=0x7f26d006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f26d00683b0 0x7f26d006a870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f26e80af000 tx=0x7f26e80a7070 comp rx=0 tx=0).stop 2024-04-27T07:04:28.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 -- 172.21.15.12:0/334387480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 msgr2=0x7f26ec1e3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 0x7f26ec1e3500 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00c450 tx=0x7f26dc00c910 comp rx=0 tx=0).stop 2024-04-27T07:04:28.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 -- 172.21.15.12:0/334387480 shutdown_connections 2024-04-27T07:04:28.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f26d00683b0 0x7f26d006a870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f26ec148810 0x7f26ec135a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26ec0b64c0 0x7f26ec1e3500 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 --2- 172.21.15.12:0/334387480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f26ec0b5ae0 0x7f26ec13a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 -- 172.21.15.12:0/334387480 >> 172.21.15.12:0/334387480 conn(0x7f26ec073c00 msgr2=0x7f26ec13ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:28.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 -- 172.21.15.12:0/334387480 shutdown_connections 2024-04-27T07:04:28.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.765+0000 7f26f3530640 1 -- 172.21.15.12:0/334387480 wait complete. 2024-04-27T07:04:28.784 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-27T07:04:28.612905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235636,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739084,"statfs":{"total":773094113280,"available":772852822016,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003972"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495786+0000","last_change":"2024-04-27T07:04:15.652893+0000","last_active":"2024-04-27T07:04:17.495786+0000","last_peered":"2024-04-27T07:04:17.495786+0000","last_clean":"2024-04-27T07:04:17.495786+0000","last_became_active":"2024-04-27T07:04:15.652155+0000","last_became_peered":"2024-04-27T07:04:15.652155+0000","last_unstale":"2024-04-27T07:04:17.495786+0000","last_undegraded":"2024-04-27T07:04:17.495786+0000","last_fullsized":"2024-04-27T07:04:17.495786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:07:07.313585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610741+0000","last_change":"2024-04-27T07:04:15.622035+0000","last_active":"2024-04-27T07:04:16.610741+0000","last_peered":"2024-04-27T07:04:16.610741+0000","last_clean":"2024-04-27T07:04:16.610741+0000","last_became_active":"2024-04-27T07:04:15.621365+0000","last_became_peered":"2024-04-27T07:04:15.621365+0000","last_unstale":"2024-04-27T07:04:16.610741+0000","last_undegraded":"2024-04-27T07:04:16.610741+0000","last_fullsized":"2024-04-27T07:04:16.610741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494175+0000","last_change":"2024-04-27T07:04:15.603108+0000","last_active":"2024-04-27T07:04:17.494175+0000","last_peered":"2024-04-27T07:04:17.494175+0000","last_clean":"2024-04-27T07:04:17.494175+0000","last_became_active":"2024-04-27T07:04:15.602672+0000","last_became_peered":"2024-04-27T07:04:15.602672+0000","last_unstale":"2024-04-27T07:04:17.494175+0000","last_undegraded":"2024-04-27T07:04:17.494175+0000","last_fullsized":"2024-04-27T07:04:17.494175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494169+0000","last_change":"2024-04-27T07:04:15.621096+0000","last_active":"2024-04-27T07:04:17.494169+0000","last_peered":"2024-04-27T07:04:17.494169+0000","last_clean":"2024-04-27T07:04:17.494169+0000","last_became_active":"2024-04-27T07:04:15.618832+0000","last_became_peered":"2024-04-27T07:04:15.618832+0000","last_unstale":"2024-04-27T07:04:17.494169+0000","last_undegraded":"2024-04-27T07:04:17.494169+0000","last_fullsized":"2024-04-27T07:04:17.494169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:08.890952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494305+0000","last_change":"2024-04-27T07:04:15.627875+0000","last_active":"2024-04-27T07:04:17.494305+0000","last_peered":"2024-04-27T07:04:17.494305+0000","last_clean":"2024-04-27T07:04:17.494305+0000","last_became_active":"2024-04-27T07:04:15.626904+0000","last_became_peered":"2024-04-27T07:04:15.626904+0000","last_unstale":"2024-04-27T07:04:17.494305+0000","last_undegraded":"2024-04-27T07:04:17.494305+0000","last_fullsized":"2024-04-27T07:04:17.494305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:11.386203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494240+0000","last_change":"2024-04-27T07:04:15.632294+0000","last_active":"2024-04-27T07:04:17.494240+0000","last_peered":"2024-04-27T07:04:17.494240+0000","last_clean":"2024-04-27T07:04:17.494240+0000","last_became_active":"2024-04-27T07:04:15.630191+0000","last_became_peered":"2024-04-27T07:04:15.630191+0000","last_unstale":"2024-04-27T07:04:17.494240+0000","last_undegraded":"2024-04-27T07:04:17.494240+0000","last_fullsized":"2024-04-27T07:04:17.494240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:19.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493321+0000","last_change":"2024-04-27T07:04:15.652899+0000","last_active":"2024-04-27T07:04:17.493321+0000","last_peered":"2024-04-27T07:04:17.493321+0000","last_clean":"2024-04-27T07:04:17.493321+0000","last_became_active":"2024-04-27T07:04:15.652383+0000","last_became_peered":"2024-04-27T07:04:15.652383+0000","last_unstale":"2024-04-27T07:04:17.493321+0000","last_undegraded":"2024-04-27T07:04:17.493321+0000","last_fullsized":"2024-04-27T07:04:17.493321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495664+0000","last_change":"2024-04-27T07:04:15.635192+0000","last_active":"2024-04-27T07:04:17.495664+0000","last_peered":"2024-04-27T07:04:17.495664+0000","last_clean":"2024-04-27T07:04:17.495664+0000","last_became_active":"2024-04-27T07:04:15.634845+0000","last_became_peered":"2024-04-27T07:04:15.634845+0000","last_unstale":"2024-04-27T07:04:17.495664+0000","last_undegraded":"2024-04-27T07:04:17.495664+0000","last_fullsized":"2024-04-27T07:04:17.495664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495704+0000","last_change":"2024-04-27T07:04:15.642203+0000","last_active":"2024-04-27T07:04:17.495704+0000","last_peered":"2024-04-27T07:04:17.495704+0000","last_clean":"2024-04-27T07:04:17.495704+0000","last_became_active":"2024-04-27T07:04:15.640450+0000","last_became_peered":"2024-04-27T07:04:15.640450+0000","last_unstale":"2024-04-27T07:04:17.495704+0000","last_undegraded":"2024-04-27T07:04:17.495704+0000","last_fullsized":"2024-04-27T07:04:17.495704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:46.491888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614056+0000","last_change":"2024-04-27T07:04:15.620338+0000","last_active":"2024-04-27T07:04:16.614056+0000","last_peered":"2024-04-27T07:04:16.614056+0000","last_clean":"2024-04-27T07:04:16.614056+0000","last_became_active":"2024-04-27T07:04:15.619521+0000","last_became_peered":"2024-04-27T07:04:15.619521+0000","last_unstale":"2024-04-27T07:04:16.614056+0000","last_undegraded":"2024-04-27T07:04:16.614056+0000","last_fullsized":"2024-04-27T07:04:16.614056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613051+0000","last_change":"2024-04-27T07:04:15.625615+0000","last_active":"2024-04-27T07:04:16.613051+0000","last_peered":"2024-04-27T07:04:16.613051+0000","last_clean":"2024-04-27T07:04:16.613051+0000","last_became_active":"2024-04-27T07:04:15.624734+0000","last_became_peered":"2024-04-27T07:04:15.624734+0000","last_unstale":"2024-04-27T07:04:16.613051+0000","last_undegraded":"2024-04-27T07:04:16.613051+0000","last_fullsized":"2024-04-27T07:04:16.613051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613805+0000","last_change":"2024-04-27T07:04:15.609501+0000","last_active":"2024-04-27T07:04:16.613805+0000","last_peered":"2024-04-27T07:04:16.613805+0000","last_clean":"2024-04-27T07:04:16.613805+0000","last_became_active":"2024-04-27T07:04:15.603709+0000","last_became_peered":"2024-04-27T07:04:15.603709+0000","last_unstale":"2024-04-27T07:04:16.613805+0000","last_undegraded":"2024-04-27T07:04:16.613805+0000","last_fullsized":"2024-04-27T07:04:16.613805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:59.982527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496346+0000","last_change":"2024-04-27T07:04:15.625180+0000","last_active":"2024-04-27T07:04:17.496346+0000","last_peered":"2024-04-27T07:04:17.496346+0000","last_clean":"2024-04-27T07:04:17.496346+0000","last_became_active":"2024-04-27T07:04:15.624748+0000","last_became_peered":"2024-04-27T07:04:15.624748+0000","last_unstale":"2024-04-27T07:04:17.496346+0000","last_undegraded":"2024-04-27T07:04:17.496346+0000","last_fullsized":"2024-04-27T07:04:17.496346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493065+0000","last_change":"2024-04-27T07:04:15.604939+0000","last_active":"2024-04-27T07:04:17.493065+0000","last_peered":"2024-04-27T07:04:17.493065+0000","last_clean":"2024-04-27T07:04:17.493065+0000","last_became_active":"2024-04-27T07:04:15.603580+0000","last_became_peered":"2024-04-27T07:04:15.603580+0000","last_unstale":"2024-04-27T07:04:17.493065+0000","last_undegraded":"2024-04-27T07:04:17.493065+0000","last_fullsized":"2024-04-27T07:04:17.493065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:45.914758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610468+0000","last_change":"2024-04-27T07:04:15.629060+0000","last_active":"2024-04-27T07:04:16.610468+0000","last_peered":"2024-04-27T07:04:16.610468+0000","last_clean":"2024-04-27T07:04:16.610468+0000","last_became_active":"2024-04-27T07:04:15.628737+0000","last_became_peered":"2024-04-27T07:04:15.628737+0000","last_unstale":"2024-04-27T07:04:16.610468+0000","last_undegraded":"2024-04-27T07:04:16.610468+0000","last_fullsized":"2024-04-27T07:04:16.610468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492723+0000","last_change":"2024-04-27T07:04:15.610164+0000","last_active":"2024-04-27T07:04:17.492723+0000","last_peered":"2024-04-27T07:04:17.492723+0000","last_clean":"2024-04-27T07:04:17.492723+0000","last_became_active":"2024-04-27T07:04:15.609290+0000","last_became_peered":"2024-04-27T07:04:15.609290+0000","last_unstale":"2024-04-27T07:04:17.492723+0000","last_undegraded":"2024-04-27T07:04:17.492723+0000","last_fullsized":"2024-04-27T07:04:17.492723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494381+0000","last_change":"2024-04-27T07:04:15.647816+0000","last_active":"2024-04-27T07:04:17.494381+0000","last_peered":"2024-04-27T07:04:17.494381+0000","last_clean":"2024-04-27T07:04:17.494381+0000","last_became_active":"2024-04-27T07:04:15.646892+0000","last_became_peered":"2024-04-27T07:04:15.646892+0000","last_unstale":"2024-04-27T07:04:17.494381+0000","last_undegraded":"2024-04-27T07:04:17.494381+0000","last_fullsized":"2024-04-27T07:04:17.494381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:01.732848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493299+0000","last_change":"2024-04-27T07:04:15.609407+0000","last_active":"2024-04-27T07:04:17.493299+0000","last_peered":"2024-04-27T07:04:17.493299+0000","last_clean":"2024-04-27T07:04:17.493299+0000","last_became_active":"2024-04-27T07:04:15.608565+0000","last_became_peered":"2024-04-27T07:04:15.608565+0000","last_unstale":"2024-04-27T07:04:17.493299+0000","last_undegraded":"2024-04-27T07:04:17.493299+0000","last_fullsized":"2024-04-27T07:04:17.493299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:37.376293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495719+0000","last_change":"2024-04-27T07:04:15.627082+0000","last_active":"2024-04-27T07:04:17.495719+0000","last_peered":"2024-04-27T07:04:17.495719+0000","last_clean":"2024-04-27T07:04:17.495719+0000","last_became_active":"2024-04-27T07:04:15.626302+0000","last_became_peered":"2024-04-27T07:04:15.626302+0000","last_unstale":"2024-04-27T07:04:17.495719+0000","last_undegraded":"2024-04-27T07:04:17.495719+0000","last_fullsized":"2024-04-27T07:04:17.495719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:01.732848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615116+0000","last_change":"2024-04-27T07:04:15.610661+0000","last_active":"2024-04-27T07:04:16.615116+0000","last_peered":"2024-04-27T07:04:16.615116+0000","last_clean":"2024-04-27T07:04:16.615116+0000","last_became_active":"2024-04-27T07:04:15.609899+0000","last_became_peered":"2024-04-27T07:04:15.609899+0000","last_unstale":"2024-04-27T07:04:16.615116+0000","last_undegraded":"2024-04-27T07:04:16.615116+0000","last_fullsized":"2024-04-27T07:04:16.615116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494321+0000","last_change":"2024-04-27T07:04:15.599714+0000","last_active":"2024-04-27T07:04:17.494321+0000","last_peered":"2024-04-27T07:04:17.494321+0000","last_clean":"2024-04-27T07:04:17.494321+0000","last_became_active":"2024-04-27T07:04:15.598855+0000","last_became_peered":"2024-04-27T07:04:15.598855+0000","last_unstale":"2024-04-27T07:04:17.494321+0000","last_undegraded":"2024-04-27T07:04:17.494321+0000","last_fullsized":"2024-04-27T07:04:17.494321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493334+0000","last_change":"2024-04-27T07:04:15.634881+0000","last_active":"2024-04-27T07:04:17.493334+0000","last_peered":"2024-04-27T07:04:17.493334+0000","last_clean":"2024-04-27T07:04:17.493334+0000","last_became_active":"2024-04-27T07:04:15.629428+0000","last_became_peered":"2024-04-27T07:04:15.629428+0000","last_unstale":"2024-04-27T07:04:17.493334+0000","last_undegraded":"2024-04-27T07:04:17.493334+0000","last_fullsized":"2024-04-27T07:04:17.493334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:40:20.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495828+0000","last_change":"2024-04-27T07:04:15.630015+0000","last_active":"2024-04-27T07:04:17.495828+0000","last_peered":"2024-04-27T07:04:17.495828+0000","last_clean":"2024-04-27T07:04:17.495828+0000","last_became_active":"2024-04-27T07:04:15.629177+0000","last_became_peered":"2024-04-27T07:04:15.629177+0000","last_unstale":"2024-04-27T07:04:17.495828+0000","last_undegraded":"2024-04-27T07:04:17.495828+0000","last_fullsized":"2024-04-27T07:04:17.495828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493333+0000","last_change":"2024-04-27T07:04:15.619427+0000","last_active":"2024-04-27T07:04:17.493333+0000","last_peered":"2024-04-27T07:04:17.493333+0000","last_clean":"2024-04-27T07:04:17.493333+0000","last_became_active":"2024-04-27T07:04:15.617987+0000","last_became_peered":"2024-04-27T07:04:15.617987+0000","last_unstale":"2024-04-27T07:04:17.493333+0000","last_undegraded":"2024-04-27T07:04:17.493333+0000","last_fullsized":"2024-04-27T07:04:17.493333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:39:04.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496704+0000","last_change":"2024-04-27T07:04:15.614547+0000","last_active":"2024-04-27T07:04:17.496704+0000","last_peered":"2024-04-27T07:04:17.496704+0000","last_clean":"2024-04-27T07:04:17.496704+0000","last_became_active":"2024-04-27T07:04:15.614210+0000","last_became_peered":"2024-04-27T07:04:15.614210+0000","last_unstale":"2024-04-27T07:04:17.496704+0000","last_undegraded":"2024-04-27T07:04:17.496704+0000","last_fullsized":"2024-04-27T07:04:17.496704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:40:20.021402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614461+0000","last_change":"2024-04-27T07:04:15.650072+0000","last_active":"2024-04-27T07:04:16.614461+0000","last_peered":"2024-04-27T07:04:16.614461+0000","last_clean":"2024-04-27T07:04:16.614461+0000","last_became_active":"2024-04-27T07:04:15.649461+0000","last_became_peered":"2024-04-27T07:04:15.649461+0000","last_unstale":"2024-04-27T07:04:16.614461+0000","last_undegraded":"2024-04-27T07:04:16.614461+0000","last_fullsized":"2024-04-27T07:04:16.614461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610542+0000","last_change":"2024-04-27T07:04:15.625742+0000","last_active":"2024-04-27T07:04:16.610542+0000","last_peered":"2024-04-27T07:04:16.610542+0000","last_clean":"2024-04-27T07:04:16.610542+0000","last_became_active":"2024-04-27T07:04:15.625049+0000","last_became_peered":"2024-04-27T07:04:15.625049+0000","last_unstale":"2024-04-27T07:04:16.610542+0000","last_undegraded":"2024-04-27T07:04:16.610542+0000","last_fullsized":"2024-04-27T07:04:16.610542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496813+0000","last_change":"2024-04-27T07:04:15.629376+0000","last_active":"2024-04-27T07:04:17.496813+0000","last_peered":"2024-04-27T07:04:17.496813+0000","last_clean":"2024-04-27T07:04:17.496813+0000","last_became_active":"2024-04-27T07:04:15.629058+0000","last_became_peered":"2024-04-27T07:04:15.629058+0000","last_unstale":"2024-04-27T07:04:17.496813+0000","last_undegraded":"2024-04-27T07:04:17.496813+0000","last_fullsized":"2024-04-27T07:04:17.496813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494539+0000","last_change":"2024-04-27T07:04:15.620883+0000","last_active":"2024-04-27T07:04:17.494539+0000","last_peered":"2024-04-27T07:04:17.494539+0000","last_clean":"2024-04-27T07:04:17.494539+0000","last_became_active":"2024-04-27T07:04:15.620549+0000","last_became_peered":"2024-04-27T07:04:15.620549+0000","last_unstale":"2024-04-27T07:04:17.494539+0000","last_undegraded":"2024-04-27T07:04:17.494539+0000","last_fullsized":"2024-04-27T07:04:17.494539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:09.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493778+0000","last_change":"2024-04-27T07:04:15.631001+0000","last_active":"2024-04-27T07:04:17.493778+0000","last_peered":"2024-04-27T07:04:17.493778+0000","last_clean":"2024-04-27T07:04:17.493778+0000","last_became_active":"2024-04-27T07:04:15.628448+0000","last_became_peered":"2024-04-27T07:04:15.628448+0000","last_unstale":"2024-04-27T07:04:17.493778+0000","last_undegraded":"2024-04-27T07:04:17.493778+0000","last_fullsized":"2024-04-27T07:04:17.493778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493518+0000","last_change":"2024-04-27T07:04:15.623687+0000","last_active":"2024-04-27T07:04:17.493518+0000","last_peered":"2024-04-27T07:04:17.493518+0000","last_clean":"2024-04-27T07:04:17.493518+0000","last_became_active":"2024-04-27T07:04:15.619092+0000","last_became_peered":"2024-04-27T07:04:15.619092+0000","last_unstale":"2024-04-27T07:04:17.493518+0000","last_undegraded":"2024-04-27T07:04:17.493518+0000","last_fullsized":"2024-04-27T07:04:17.493518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:50.960522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612331+0000","last_change":"2024-04-27T07:04:15.601950+0000","last_active":"2024-04-27T07:04:16.612331+0000","last_peered":"2024-04-27T07:04:16.612331+0000","last_clean":"2024-04-27T07:04:16.612331+0000","last_became_active":"2024-04-27T07:04:15.601144+0000","last_became_peered":"2024-04-27T07:04:15.601144+0000","last_unstale":"2024-04-27T07:04:16.612331+0000","last_undegraded":"2024-04-27T07:04:16.612331+0000","last_fullsized":"2024-04-27T07:04:16.612331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:55.937844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613553+0000","last_change":"2024-04-27T07:04:15.631943+0000","last_active":"2024-04-27T07:04:16.613553+0000","last_peered":"2024-04-27T07:04:16.613553+0000","last_clean":"2024-04-27T07:04:16.613553+0000","last_became_active":"2024-04-27T07:04:15.619207+0000","last_became_peered":"2024-04-27T07:04:15.619207+0000","last_unstale":"2024-04-27T07:04:16.613553+0000","last_undegraded":"2024-04-27T07:04:16.613553+0000","last_fullsized":"2024-04-27T07:04:16.613553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:58.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496502+0000","last_change":"2024-04-27T07:04:14.615664+0000","last_active":"2024-04-27T07:04:17.496502+0000","last_peered":"2024-04-27T07:04:17.496502+0000","last_clean":"2024-04-27T07:04:17.496502+0000","last_became_active":"2024-04-27T07:04:14.615220+0000","last_became_peered":"2024-04-27T07:04:14.615220+0000","last_unstale":"2024-04-27T07:04:17.496502+0000","last_undegraded":"2024-04-27T07:04:17.496502+0000","last_fullsized":"2024-04-27T07:04:17.496502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494747+0000","last_change":"2024-04-27T07:04:15.619876+0000","last_active":"2024-04-27T07:04:17.494747+0000","last_peered":"2024-04-27T07:04:17.494747+0000","last_clean":"2024-04-27T07:04:17.494747+0000","last_became_active":"2024-04-27T07:04:15.619131+0000","last_became_peered":"2024-04-27T07:04:15.619131+0000","last_unstale":"2024-04-27T07:04:17.494747+0000","last_undegraded":"2024-04-27T07:04:17.494747+0000","last_fullsized":"2024-04-27T07:04:17.494747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614112+0000","last_change":"2024-04-27T07:04:14.611792+0000","last_active":"2024-04-27T07:04:16.614112+0000","last_peered":"2024-04-27T07:04:16.614112+0000","last_clean":"2024-04-27T07:04:16.614112+0000","last_became_active":"2024-04-27T07:04:14.609221+0000","last_became_peered":"2024-04-27T07:04:14.609221+0000","last_unstale":"2024-04-27T07:04:16.614112+0000","last_undegraded":"2024-04-27T07:04:16.614112+0000","last_fullsized":"2024-04-27T07:04:16.614112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495969+0000","last_change":"2024-04-27T07:04:14.608168+0000","last_active":"2024-04-27T07:04:17.495969+0000","last_peered":"2024-04-27T07:04:17.495969+0000","last_clean":"2024-04-27T07:04:17.495969+0000","last_became_active":"2024-04-27T07:04:14.607735+0000","last_became_peered":"2024-04-27T07:04:14.607735+0000","last_unstale":"2024-04-27T07:04:17.495969+0000","last_undegraded":"2024-04-27T07:04:17.495969+0000","last_fullsized":"2024-04-27T07:04:17.495969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:06.395968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495767+0000","last_change":"2024-04-27T07:04:15.612016+0000","last_active":"2024-04-27T07:04:17.495767+0000","last_peered":"2024-04-27T07:04:17.495767+0000","last_clean":"2024-04-27T07:04:17.495767+0000","last_became_active":"2024-04-27T07:04:15.611658+0000","last_became_peered":"2024-04-27T07:04:15.611658+0000","last_unstale":"2024-04-27T07:04:17.495767+0000","last_undegraded":"2024-04-27T07:04:17.495767+0000","last_fullsized":"2024-04-27T07:04:17.495767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492906+0000","last_change":"2024-04-27T07:04:14.620476+0000","last_active":"2024-04-27T07:04:17.492906+0000","last_peered":"2024-04-27T07:04:17.492906+0000","last_clean":"2024-04-27T07:04:17.492906+0000","last_became_active":"2024-04-27T07:04:14.617893+0000","last_became_peered":"2024-04-27T07:04:14.617893+0000","last_unstale":"2024-04-27T07:04:17.492906+0000","last_undegraded":"2024-04-27T07:04:17.492906+0000","last_fullsized":"2024-04-27T07:04:17.492906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610222+0000","last_change":"2024-04-27T07:04:15.624591+0000","last_active":"2024-04-27T07:04:16.610222+0000","last_peered":"2024-04-27T07:04:16.610222+0000","last_clean":"2024-04-27T07:04:16.610222+0000","last_became_active":"2024-04-27T07:04:15.624299+0000","last_became_peered":"2024-04-27T07:04:15.624299+0000","last_unstale":"2024-04-27T07:04:16.610222+0000","last_undegraded":"2024-04-27T07:04:16.610222+0000","last_fullsized":"2024-04-27T07:04:16.610222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614771+0000","last_change":"2024-04-27T07:04:14.621378+0000","last_active":"2024-04-27T07:04:16.614771+0000","last_peered":"2024-04-27T07:04:16.614771+0000","last_clean":"2024-04-27T07:04:16.614771+0000","last_became_active":"2024-04-27T07:04:14.620678+0000","last_became_peered":"2024-04-27T07:04:14.620678+0000","last_unstale":"2024-04-27T07:04:16.614771+0000","last_undegraded":"2024-04-27T07:04:16.614771+0000","last_fullsized":"2024-04-27T07:04:16.614771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493724+0000","last_change":"2024-04-27T07:04:15.647396+0000","last_active":"2024-04-27T07:04:17.493724+0000","last_peered":"2024-04-27T07:04:17.493724+0000","last_clean":"2024-04-27T07:04:17.493724+0000","last_became_active":"2024-04-27T07:04:15.646774+0000","last_became_peered":"2024-04-27T07:04:15.646774+0000","last_unstale":"2024-04-27T07:04:17.493724+0000","last_undegraded":"2024-04-27T07:04:17.493724+0000","last_fullsized":"2024-04-27T07:04:17.493724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:04.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493229+0000","last_change":"2024-04-27T07:04:14.621935+0000","last_active":"2024-04-27T07:04:17.493229+0000","last_peered":"2024-04-27T07:04:17.493229+0000","last_clean":"2024-04-27T07:04:17.493229+0000","last_became_active":"2024-04-27T07:04:14.620120+0000","last_became_peered":"2024-04-27T07:04:14.620120+0000","last_unstale":"2024-04-27T07:04:17.493229+0000","last_undegraded":"2024-04-27T07:04:17.493229+0000","last_fullsized":"2024-04-27T07:04:17.493229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494443+0000","last_change":"2024-04-27T07:04:15.626451+0000","last_active":"2024-04-27T07:04:17.494443+0000","last_peered":"2024-04-27T07:04:17.494443+0000","last_clean":"2024-04-27T07:04:17.494443+0000","last_became_active":"2024-04-27T07:04:15.626147+0000","last_became_peered":"2024-04-27T07:04:15.626147+0000","last_unstale":"2024-04-27T07:04:17.494443+0000","last_undegraded":"2024-04-27T07:04:17.494443+0000","last_fullsized":"2024-04-27T07:04:17.494443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610433+0000","last_change":"2024-04-27T07:04:14.619618+0000","last_active":"2024-04-27T07:04:16.610433+0000","last_peered":"2024-04-27T07:04:16.610433+0000","last_clean":"2024-04-27T07:04:16.610433+0000","last_became_active":"2024-04-27T07:04:14.619155+0000","last_became_peered":"2024-04-27T07:04:14.619155+0000","last_unstale":"2024-04-27T07:04:16.610433+0000","last_undegraded":"2024-04-27T07:04:16.610433+0000","last_fullsized":"2024-04-27T07:04:16.610433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:11:57.110182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614234+0000","last_change":"2024-04-27T07:04:15.628583+0000","last_active":"2024-04-27T07:04:16.614234+0000","last_peered":"2024-04-27T07:04:16.614234+0000","last_clean":"2024-04-27T07:04:16.614234+0000","last_became_active":"2024-04-27T07:04:15.621801+0000","last_became_peered":"2024-04-27T07:04:15.621801+0000","last_unstale":"2024-04-27T07:04:16.614234+0000","last_undegraded":"2024-04-27T07:04:16.614234+0000","last_fullsized":"2024-04-27T07:04:16.614234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:37.376293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615199+0000","last_change":"2024-04-27T07:04:14.609138+0000","last_active":"2024-04-27T07:04:16.615199+0000","last_peered":"2024-04-27T07:04:16.615199+0000","last_clean":"2024-04-27T07:04:16.615199+0000","last_became_active":"2024-04-27T07:04:14.608374+0000","last_became_peered":"2024-04-27T07:04:14.608374+0000","last_unstale":"2024-04-27T07:04:16.615199+0000","last_undegraded":"2024-04-27T07:04:16.615199+0000","last_fullsized":"2024-04-27T07:04:16.615199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495184+0000","last_change":"2024-04-27T07:04:15.651834+0000","last_active":"2024-04-27T07:04:17.495184+0000","last_peered":"2024-04-27T07:04:17.495184+0000","last_clean":"2024-04-27T07:04:17.495184+0000","last_became_active":"2024-04-27T07:04:15.651286+0000","last_became_peered":"2024-04-27T07:04:15.651286+0000","last_unstale":"2024-04-27T07:04:17.495184+0000","last_undegraded":"2024-04-27T07:04:17.495184+0000","last_fullsized":"2024-04-27T07:04:17.495184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:00:38.115028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494797+0000","last_change":"2024-04-27T07:04:14.616280+0000","last_active":"2024-04-27T07:04:17.494797+0000","last_peered":"2024-04-27T07:04:17.494797+0000","last_clean":"2024-04-27T07:04:17.494797+0000","last_became_active":"2024-04-27T07:04:14.615920+0000","last_became_peered":"2024-04-27T07:04:14.615920+0000","last_unstale":"2024-04-27T07:04:17.494797+0000","last_undegraded":"2024-04-27T07:04:17.494797+0000","last_fullsized":"2024-04-27T07:04:17.494797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:06.395968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493937+0000","last_change":"2024-04-27T07:04:15.647042+0000","last_active":"2024-04-27T07:04:17.493937+0000","last_peered":"2024-04-27T07:04:17.493937+0000","last_clean":"2024-04-27T07:04:17.493937+0000","last_became_active":"2024-04-27T07:04:15.646683+0000","last_became_peered":"2024-04-27T07:04:15.646683+0000","last_unstale":"2024-04-27T07:04:17.493937+0000","last_undegraded":"2024-04-27T07:04:17.493937+0000","last_fullsized":"2024-04-27T07:04:17.493937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:05:55.398463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493914+0000","last_change":"2024-04-27T07:04:14.620284+0000","last_active":"2024-04-27T07:04:17.493914+0000","last_peered":"2024-04-27T07:04:17.493914+0000","last_clean":"2024-04-27T07:04:17.493914+0000","last_became_active":"2024-04-27T07:04:14.619379+0000","last_became_peered":"2024-04-27T07:04:14.619379+0000","last_unstale":"2024-04-27T07:04:17.493914+0000","last_undegraded":"2024-04-27T07:04:17.493914+0000","last_fullsized":"2024-04-27T07:04:17.493914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495014+0000","last_change":"2024-04-27T07:04:15.635161+0000","last_active":"2024-04-27T07:04:17.495014+0000","last_peered":"2024-04-27T07:04:17.495014+0000","last_clean":"2024-04-27T07:04:17.495014+0000","last_became_active":"2024-04-27T07:04:15.634423+0000","last_became_peered":"2024-04-27T07:04:15.634423+0000","last_unstale":"2024-04-27T07:04:17.495014+0000","last_undegraded":"2024-04-27T07:04:17.495014+0000","last_fullsized":"2024-04-27T07:04:17.495014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:17.226045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612693+0000","last_change":"2024-04-27T07:04:14.612975+0000","last_active":"2024-04-27T07:04:16.612693+0000","last_peered":"2024-04-27T07:04:16.612693+0000","last_clean":"2024-04-27T07:04:16.612693+0000","last_became_active":"2024-04-27T07:04:14.608937+0000","last_became_peered":"2024-04-27T07:04:14.608937+0000","last_unstale":"2024-04-27T07:04:16.612693+0000","last_undegraded":"2024-04-27T07:04:16.612693+0000","last_fullsized":"2024-04-27T07:04:16.612693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.612667+0000","last_change":"2024-04-27T07:04:15.601024+0000","last_active":"2024-04-27T07:04:16.612667+0000","last_peered":"2024-04-27T07:04:16.612667+0000","last_clean":"2024-04-27T07:04:16.612667+0000","last_became_active":"2024-04-27T07:04:15.600245+0000","last_became_peered":"2024-04-27T07:04:15.600245+0000","last_unstale":"2024-04-27T07:04:16.612667+0000","last_undegraded":"2024-04-27T07:04:16.612667+0000","last_fullsized":"2024-04-27T07:04:16.612667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:01:08.227386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492667+0000","last_change":"2024-04-27T07:04:14.594793+0000","last_active":"2024-04-27T07:04:17.492667+0000","last_peered":"2024-04-27T07:04:17.492667+0000","last_clean":"2024-04-27T07:04:17.492667+0000","last_became_active":"2024-04-27T07:04:14.594022+0000","last_became_peered":"2024-04-27T07:04:14.594022+0000","last_unstale":"2024-04-27T07:04:17.492667+0000","last_undegraded":"2024-04-27T07:04:17.492667+0000","last_fullsized":"2024-04-27T07:04:17.492667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:01.729925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493727+0000","last_change":"2024-04-27T07:04:15.639932+0000","last_active":"2024-04-27T07:04:17.493727+0000","last_peered":"2024-04-27T07:04:17.493727+0000","last_clean":"2024-04-27T07:04:17.493727+0000","last_became_active":"2024-04-27T07:04:15.639192+0000","last_became_peered":"2024-04-27T07:04:15.639192+0000","last_unstale":"2024-04-27T07:04:17.493727+0000","last_undegraded":"2024-04-27T07:04:17.493727+0000","last_fullsized":"2024-04-27T07:04:17.493727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614105+0000","last_change":"2024-04-27T07:04:14.605930+0000","last_active":"2024-04-27T07:04:16.614105+0000","last_peered":"2024-04-27T07:04:16.614105+0000","last_clean":"2024-04-27T07:04:16.614105+0000","last_became_active":"2024-04-27T07:04:14.604957+0000","last_became_peered":"2024-04-27T07:04:14.604957+0000","last_unstale":"2024-04-27T07:04:16.614105+0000","last_undegraded":"2024-04-27T07:04:16.614105+0000","last_fullsized":"2024-04-27T07:04:16.614105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:15.465739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495558+0000","last_change":"2024-04-27T07:04:15.635261+0000","last_active":"2024-04-27T07:04:17.495558+0000","last_peered":"2024-04-27T07:04:17.495558+0000","last_clean":"2024-04-27T07:04:17.495558+0000","last_became_active":"2024-04-27T07:04:15.634020+0000","last_became_peered":"2024-04-27T07:04:15.634020+0000","last_unstale":"2024-04-27T07:04:17.495558+0000","last_undegraded":"2024-04-27T07:04:17.495558+0000","last_fullsized":"2024-04-27T07:04:17.495558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:08.890952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493458+0000","last_change":"2024-04-27T07:04:14.600680+0000","last_active":"2024-04-27T07:04:17.493458+0000","last_peered":"2024-04-27T07:04:17.493458+0000","last_clean":"2024-04-27T07:04:17.493458+0000","last_became_active":"2024-04-27T07:04:14.600233+0000","last_became_peered":"2024-04-27T07:04:14.600233+0000","last_unstale":"2024-04-27T07:04:17.493458+0000","last_undegraded":"2024-04-27T07:04:17.493458+0000","last_fullsized":"2024-04-27T07:04:17.493458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495918+0000","last_change":"2024-04-27T07:04:15.643279+0000","last_active":"2024-04-27T07:04:17.495918+0000","last_peered":"2024-04-27T07:04:17.495918+0000","last_clean":"2024-04-27T07:04:17.495918+0000","last_became_active":"2024-04-27T07:04:15.641303+0000","last_became_peered":"2024-04-27T07:04:15.641303+0000","last_unstale":"2024-04-27T07:04:17.495918+0000","last_undegraded":"2024-04-27T07:04:17.495918+0000","last_fullsized":"2024-04-27T07:04:17.495918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:02:46.002992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610599+0000","last_change":"2024-04-27T07:04:14.617808+0000","last_active":"2024-04-27T07:04:16.610599+0000","last_peered":"2024-04-27T07:04:16.610599+0000","last_clean":"2024-04-27T07:04:16.610599+0000","last_became_active":"2024-04-27T07:04:14.617505+0000","last_became_peered":"2024-04-27T07:04:14.617505+0000","last_unstale":"2024-04-27T07:04:16.610599+0000","last_undegraded":"2024-04-27T07:04:16.610599+0000","last_fullsized":"2024-04-27T07:04:16.610599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494318+0000","last_change":"2024-04-27T07:04:15.619586+0000","last_active":"2024-04-27T07:04:17.494318+0000","last_peered":"2024-04-27T07:04:17.494318+0000","last_clean":"2024-04-27T07:04:17.494318+0000","last_became_active":"2024-04-27T07:04:15.617842+0000","last_became_peered":"2024-04-27T07:04:15.617842+0000","last_unstale":"2024-04-27T07:04:17.494318+0000","last_undegraded":"2024-04-27T07:04:17.494318+0000","last_fullsized":"2024-04-27T07:04:17.494318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:46.491888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493601+0000","last_change":"2024-04-27T07:04:14.619149+0000","last_active":"2024-04-27T07:04:17.493601+0000","last_peered":"2024-04-27T07:04:17.493601+0000","last_clean":"2024-04-27T07:04:17.493601+0000","last_became_active":"2024-04-27T07:04:14.618660+0000","last_became_peered":"2024-04-27T07:04:14.618660+0000","last_unstale":"2024-04-27T07:04:17.493601+0000","last_undegraded":"2024-04-27T07:04:17.493601+0000","last_fullsized":"2024-04-27T07:04:17.493601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495600+0000","last_change":"2024-04-27T07:04:15.640980+0000","last_active":"2024-04-27T07:04:17.495600+0000","last_peered":"2024-04-27T07:04:17.495600+0000","last_clean":"2024-04-27T07:04:17.495600+0000","last_became_active":"2024-04-27T07:04:15.640289+0000","last_became_peered":"2024-04-27T07:04:15.640289+0000","last_unstale":"2024-04-27T07:04:17.495600+0000","last_undegraded":"2024-04-27T07:04:17.495600+0000","last_fullsized":"2024-04-27T07:04:17.495600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:19.137712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492606+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:04:17.492606+0000","last_peered":"2024-04-27T07:04:17.492606+0000","last_clean":"2024-04-27T07:04:17.492606+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:04:17.492606+0000","last_undegraded":"2024-04-27T07:04:17.492606+0000","last_fullsized":"2024-04-27T07:04:17.492606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493168+0000","last_change":"2024-04-27T07:04:14.621914+0000","last_active":"2024-04-27T07:04:17.493168+0000","last_peered":"2024-04-27T07:04:17.493168+0000","last_clean":"2024-04-27T07:04:17.493168+0000","last_became_active":"2024-04-27T07:04:14.620901+0000","last_became_peered":"2024-04-27T07:04:14.620901+0000","last_unstale":"2024-04-27T07:04:17.493168+0000","last_undegraded":"2024-04-27T07:04:17.493168+0000","last_fullsized":"2024-04-27T07:04:17.493168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:35:13.065177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496303+0000","last_change":"2024-04-27T07:04:15.635276+0000","last_active":"2024-04-27T07:04:17.496303+0000","last_peered":"2024-04-27T07:04:17.496303+0000","last_clean":"2024-04-27T07:04:17.496303+0000","last_became_active":"2024-04-27T07:04:15.634946+0000","last_became_peered":"2024-04-27T07:04:15.634946+0000","last_unstale":"2024-04-27T07:04:17.496303+0000","last_undegraded":"2024-04-27T07:04:17.496303+0000","last_fullsized":"2024-04-27T07:04:17.496303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492322+0000","last_change":"2024-04-27T07:04:14.628608+0000","last_active":"2024-04-27T07:04:17.492322+0000","last_peered":"2024-04-27T07:04:17.492322+0000","last_clean":"2024-04-27T07:04:17.492322+0000","last_became_active":"2024-04-27T07:04:14.626795+0000","last_became_peered":"2024-04-27T07:04:14.626795+0000","last_unstale":"2024-04-27T07:04:17.492322+0000","last_undegraded":"2024-04-27T07:04:17.492322+0000","last_fullsized":"2024-04-27T07:04:17.492322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:32:26.049320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493125+0000","last_change":"2024-04-27T07:04:15.604705+0000","last_active":"2024-04-27T07:04:17.493125+0000","last_peered":"2024-04-27T07:04:17.493125+0000","last_clean":"2024-04-27T07:04:17.493125+0000","last_became_active":"2024-04-27T07:04:15.603226+0000","last_became_peered":"2024-04-27T07:04:15.603226+0000","last_unstale":"2024-04-27T07:04:17.493125+0000","last_undegraded":"2024-04-27T07:04:17.493125+0000","last_fullsized":"2024-04-27T07:04:17.493125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:24.246666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492975+0000","last_change":"2024-04-27T07:04:14.601261+0000","last_active":"2024-04-27T07:04:17.492975+0000","last_peered":"2024-04-27T07:04:17.492975+0000","last_clean":"2024-04-27T07:04:17.492975+0000","last_became_active":"2024-04-27T07:04:14.600555+0000","last_became_peered":"2024-04-27T07:04:14.600555+0000","last_unstale":"2024-04-27T07:04:17.492975+0000","last_undegraded":"2024-04-27T07:04:17.492975+0000","last_fullsized":"2024-04-27T07:04:17.492975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610538+0000","last_change":"2024-04-27T07:04:15.632176+0000","last_active":"2024-04-27T07:04:16.610538+0000","last_peered":"2024-04-27T07:04:16.610538+0000","last_clean":"2024-04-27T07:04:16.610538+0000","last_became_active":"2024-04-27T07:04:15.631861+0000","last_became_peered":"2024-04-27T07:04:15.631861+0000","last_unstale":"2024-04-27T07:04:16.610538+0000","last_undegraded":"2024-04-27T07:04:16.610538+0000","last_fullsized":"2024-04-27T07:04:16.610538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494374+0000","last_change":"2024-04-27T07:04:14.623584+0000","last_active":"2024-04-27T07:04:17.494374+0000","last_peered":"2024-04-27T07:04:17.494374+0000","last_clean":"2024-04-27T07:04:17.494374+0000","last_became_active":"2024-04-27T07:04:14.620709+0000","last_became_peered":"2024-04-27T07:04:14.620709+0000","last_unstale":"2024-04-27T07:04:17.494374+0000","last_undegraded":"2024-04-27T07:04:17.494374+0000","last_fullsized":"2024-04-27T07:04:17.494374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.614757+0000","last_change":"2024-04-27T07:04:15.633827+0000","last_active":"2024-04-27T07:04:16.614757+0000","last_peered":"2024-04-27T07:04:16.614757+0000","last_clean":"2024-04-27T07:04:16.614757+0000","last_became_active":"2024-04-27T07:04:15.632912+0000","last_became_peered":"2024-04-27T07:04:15.632912+0000","last_unstale":"2024-04-27T07:04:16.614757+0000","last_undegraded":"2024-04-27T07:04:16.614757+0000","last_fullsized":"2024-04-27T07:04:16.614757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:50.960522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610382+0000","last_change":"2024-04-27T07:04:14.603584+0000","last_active":"2024-04-27T07:04:16.610382+0000","last_peered":"2024-04-27T07:04:16.610382+0000","last_clean":"2024-04-27T07:04:16.610382+0000","last_became_active":"2024-04-27T07:04:14.602186+0000","last_became_peered":"2024-04-27T07:04:14.602186+0000","last_unstale":"2024-04-27T07:04:16.610382+0000","last_undegraded":"2024-04-27T07:04:16.610382+0000","last_fullsized":"2024-04-27T07:04:16.610382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:08.717704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494048+0000","last_change":"2024-04-27T07:04:15.620627+0000","last_active":"2024-04-27T07:04:17.494048+0000","last_peered":"2024-04-27T07:04:17.494048+0000","last_clean":"2024-04-27T07:04:17.494048+0000","last_became_active":"2024-04-27T07:04:15.619725+0000","last_became_peered":"2024-04-27T07:04:15.619725+0000","last_unstale":"2024-04-27T07:04:17.494048+0000","last_undegraded":"2024-04-27T07:04:17.494048+0000","last_fullsized":"2024-04-27T07:04:17.494048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:17.226045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495737+0000","last_change":"2024-04-27T07:04:14.630298+0000","last_active":"2024-04-27T07:04:17.495737+0000","last_peered":"2024-04-27T07:04:17.495737+0000","last_clean":"2024-04-27T07:04:17.495737+0000","last_became_active":"2024-04-27T07:04:14.629539+0000","last_became_peered":"2024-04-27T07:04:14.629539+0000","last_unstale":"2024-04-27T07:04:17.495737+0000","last_undegraded":"2024-04-27T07:04:17.495737+0000","last_fullsized":"2024-04-27T07:04:17.495737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:44.902463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610300+0000","last_change":"2024-04-27T07:04:15.604054+0000","last_active":"2024-04-27T07:04:16.610300+0000","last_peered":"2024-04-27T07:04:16.610300+0000","last_clean":"2024-04-27T07:04:16.610300+0000","last_became_active":"2024-04-27T07:04:15.603243+0000","last_became_peered":"2024-04-27T07:04:15.603243+0000","last_unstale":"2024-04-27T07:04:16.610300+0000","last_undegraded":"2024-04-27T07:04:16.610300+0000","last_fullsized":"2024-04-27T07:04:16.610300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:36:18.161049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610793+0000","last_change":"2024-04-27T07:04:14.614799+0000","last_active":"2024-04-27T07:04:16.610793+0000","last_peered":"2024-04-27T07:04:16.610793+0000","last_clean":"2024-04-27T07:04:16.610793+0000","last_became_active":"2024-04-27T07:04:14.614348+0000","last_became_peered":"2024-04-27T07:04:14.614348+0000","last_unstale":"2024-04-27T07:04:16.610793+0000","last_undegraded":"2024-04-27T07:04:16.610793+0000","last_fullsized":"2024-04-27T07:04:16.610793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:49.948227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.497067+0000","last_change":"2024-04-27T07:04:15.649374+0000","last_active":"2024-04-27T07:04:17.497067+0000","last_peered":"2024-04-27T07:04:17.497067+0000","last_clean":"2024-04-27T07:04:17.497067+0000","last_became_active":"2024-04-27T07:04:15.649079+0000","last_became_peered":"2024-04-27T07:04:15.649079+0000","last_unstale":"2024-04-27T07:04:17.497067+0000","last_undegraded":"2024-04-27T07:04:17.497067+0000","last_fullsized":"2024-04-27T07:04:17.497067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:41:44.501496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.615045+0000","last_change":"2024-04-27T07:04:14.630929+0000","last_active":"2024-04-27T07:04:16.615045+0000","last_peered":"2024-04-27T07:04:16.615045+0000","last_clean":"2024-04-27T07:04:16.615045+0000","last_became_active":"2024-04-27T07:04:14.630345+0000","last_became_peered":"2024-04-27T07:04:14.630345+0000","last_unstale":"2024-04-27T07:04:16.615045+0000","last_undegraded":"2024-04-27T07:04:16.615045+0000","last_fullsized":"2024-04-27T07:04:16.615045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494516+0000","last_change":"2024-04-27T07:04:15.605948+0000","last_active":"2024-04-27T07:04:17.494516+0000","last_peered":"2024-04-27T07:04:17.494516+0000","last_clean":"2024-04-27T07:04:17.494516+0000","last_became_active":"2024-04-27T07:04:15.603763+0000","last_became_peered":"2024-04-27T07:04:15.603763+0000","last_unstale":"2024-04-27T07:04:17.494516+0000","last_undegraded":"2024-04-27T07:04:17.494516+0000","last_fullsized":"2024-04-27T07:04:17.494516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:05.477126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.492302+0000","last_change":"2024-04-27T07:04:14.628661+0000","last_active":"2024-04-27T07:04:17.492302+0000","last_peered":"2024-04-27T07:04:17.492302+0000","last_clean":"2024-04-27T07:04:17.492302+0000","last_became_active":"2024-04-27T07:04:14.626731+0000","last_became_peered":"2024-04-27T07:04:14.626731+0000","last_unstale":"2024-04-27T07:04:17.492302+0000","last_undegraded":"2024-04-27T07:04:17.492302+0000","last_fullsized":"2024-04-27T07:04:17.492302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:58.970232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613464+0000","last_change":"2024-04-27T07:04:15.608033+0000","last_active":"2024-04-27T07:04:16.613464+0000","last_peered":"2024-04-27T07:04:16.613464+0000","last_clean":"2024-04-27T07:04:16.613464+0000","last_became_active":"2024-04-27T07:04:15.607219+0000","last_became_peered":"2024-04-27T07:04:15.607219+0000","last_unstale":"2024-04-27T07:04:16.613464+0000","last_undegraded":"2024-04-27T07:04:16.613464+0000","last_fullsized":"2024-04-27T07:04:16.613464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:02.140237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610281+0000","last_change":"2024-04-27T07:04:14.613406+0000","last_active":"2024-04-27T07:04:16.610281+0000","last_peered":"2024-04-27T07:04:16.610281+0000","last_clean":"2024-04-27T07:04:16.610281+0000","last_became_active":"2024-04-27T07:04:14.613017+0000","last_became_peered":"2024-04-27T07:04:14.613017+0000","last_unstale":"2024-04-27T07:04:16.610281+0000","last_undegraded":"2024-04-27T07:04:16.610281+0000","last_fullsized":"2024-04-27T07:04:16.610281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496138+0000","last_change":"2024-04-27T07:04:15.617597+0000","last_active":"2024-04-27T07:04:17.496138+0000","last_peered":"2024-04-27T07:04:17.496138+0000","last_clean":"2024-04-27T07:04:17.496138+0000","last_became_active":"2024-04-27T07:04:15.616122+0000","last_became_peered":"2024-04-27T07:04:15.616122+0000","last_unstale":"2024-04-27T07:04:17.496138+0000","last_undegraded":"2024-04-27T07:04:17.496138+0000","last_fullsized":"2024-04-27T07:04:17.496138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.610542+0000","last_change":"2024-04-27T07:04:14.604022+0000","last_active":"2024-04-27T07:04:16.610542+0000","last_peered":"2024-04-27T07:04:16.610542+0000","last_clean":"2024-04-27T07:04:16.610542+0000","last_became_active":"2024-04-27T07:04:14.603308+0000","last_became_peered":"2024-04-27T07:04:14.603308+0000","last_unstale":"2024-04-27T07:04:16.610542+0000","last_undegraded":"2024-04-27T07:04:16.610542+0000","last_fullsized":"2024-04-27T07:04:16.610542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:01.127942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496561+0000","last_change":"2024-04-27T07:04:15.620378+0000","last_active":"2024-04-27T07:04:17.496561+0000","last_peered":"2024-04-27T07:04:17.496561+0000","last_clean":"2024-04-27T07:04:17.496561+0000","last_became_active":"2024-04-27T07:04:15.620100+0000","last_became_peered":"2024-04-27T07:04:15.620100+0000","last_unstale":"2024-04-27T07:04:17.496561+0000","last_undegraded":"2024-04-27T07:04:17.496561+0000","last_fullsized":"2024-04-27T07:04:17.496561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:22:19.294793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494886+0000","last_change":"2024-04-27T07:04:14.629851+0000","last_active":"2024-04-27T07:04:17.494886+0000","last_peered":"2024-04-27T07:04:17.494886+0000","last_clean":"2024-04-27T07:04:17.494886+0000","last_became_active":"2024-04-27T07:04:14.629063+0000","last_became_peered":"2024-04-27T07:04:14.629063+0000","last_unstale":"2024-04-27T07:04:17.494886+0000","last_undegraded":"2024-04-27T07:04:17.494886+0000","last_fullsized":"2024-04-27T07:04:17.494886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:29:49.948227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:16.613962+0000","last_change":"2024-04-27T07:04:15.616289+0000","last_active":"2024-04-27T07:04:16.613962+0000","last_peered":"2024-04-27T07:04:16.613962+0000","last_clean":"2024-04-27T07:04:16.613962+0000","last_became_active":"2024-04-27T07:04:15.615386+0000","last_became_peered":"2024-04-27T07:04:15.615386+0000","last_unstale":"2024-04-27T07:04:16.613962+0000","last_undegraded":"2024-04-27T07:04:16.613962+0000","last_fullsized":"2024-04-27T07:04:16.613962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.493841+0000","last_change":"2024-04-27T07:04:15.646050+0000","last_active":"2024-04-27T07:04:17.493841+0000","last_peered":"2024-04-27T07:04:17.493841+0000","last_clean":"2024-04-27T07:04:17.493841+0000","last_became_active":"2024-04-27T07:04:15.645284+0000","last_became_peered":"2024-04-27T07:04:15.645284+0000","last_unstale":"2024-04-27T07:04:17.493841+0000","last_undegraded":"2024-04-27T07:04:17.493841+0000","last_fullsized":"2024-04-27T07:04:17.493841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:55:11.386203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494836+0000","last_change":"2024-04-27T07:04:14.629860+0000","last_active":"2024-04-27T07:04:17.494836+0000","last_peered":"2024-04-27T07:04:17.494836+0000","last_clean":"2024-04-27T07:04:17.494836+0000","last_became_active":"2024-04-27T07:04:14.629138+0000","last_became_peered":"2024-04-27T07:04:14.629138+0000","last_unstale":"2024-04-27T07:04:17.494836+0000","last_undegraded":"2024-04-27T07:04:17.494836+0000","last_fullsized":"2024-04-27T07:04:17.494836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:33:36.363998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494061+0000","last_change":"2024-04-27T07:04:15.638833+0000","last_active":"2024-04-27T07:04:17.494061+0000","last_peered":"2024-04-27T07:04:17.494061+0000","last_clean":"2024-04-27T07:04:17.494061+0000","last_became_active":"2024-04-27T07:04:15.638192+0000","last_became_peered":"2024-04-27T07:04:15.638192+0000","last_unstale":"2024-04-27T07:04:17.494061+0000","last_undegraded":"2024-04-27T07:04:17.494061+0000","last_fullsized":"2024-04-27T07:04:17.494061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:00:38.115028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.495394+0000","last_change":"2024-04-27T07:04:14.632528+0000","last_active":"2024-04-27T07:04:17.495394+0000","last_peered":"2024-04-27T07:04:17.495394+0000","last_clean":"2024-04-27T07:04:17.495394+0000","last_became_active":"2024-04-27T07:04:14.631089+0000","last_became_peered":"2024-04-27T07:04:14.631089+0000","last_unstale":"2024-04-27T07:04:17.495394+0000","last_undegraded":"2024-04-27T07:04:17.495394+0000","last_fullsized":"2024-04-27T07:04:17.495394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T19:03:50.734904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494459+0000","last_change":"2024-04-27T07:04:15.626754+0000","last_active":"2024-04-27T07:04:17.494459+0000","last_peered":"2024-04-27T07:04:17.494459+0000","last_clean":"2024-04-27T07:04:17.494459+0000","last_became_active":"2024-04-27T07:04:15.626499+0000","last_became_peered":"2024-04-27T07:04:15.626499+0000","last_unstale":"2024-04-27T07:04:17.494459+0000","last_undegraded":"2024-04-27T07:04:17.494459+0000","last_fullsized":"2024-04-27T07:04:17.494459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:33:27.043564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496732+0000","last_change":"2024-04-27T07:04:14.598918+0000","last_active":"2024-04-27T07:04:17.496732+0000","last_peered":"2024-04-27T07:04:17.496732+0000","last_clean":"2024-04-27T07:04:17.496732+0000","last_became_active":"2024-04-27T07:04:14.598504+0000","last_became_peered":"2024-04-27T07:04:14.598504+0000","last_unstale":"2024-04-27T07:04:17.496732+0000","last_undegraded":"2024-04-27T07:04:17.496732+0000","last_fullsized":"2024-04-27T07:04:17.496732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:01.729925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.496574+0000","last_change":"2024-04-27T07:04:15.625377+0000","last_active":"2024-04-27T07:04:17.496574+0000","last_peered":"2024-04-27T07:04:17.496574+0000","last_clean":"2024-04-27T07:04:17.496574+0000","last_became_active":"2024-04-27T07:04:15.625075+0000","last_became_peered":"2024-04-27T07:04:15.625075+0000","last_unstale":"2024-04-27T07:04:17.496574+0000","last_undegraded":"2024-04-27T07:04:17.496574+0000","last_fullsized":"2024-04-27T07:04:17.496574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:14.566367+0000","last_clean_scrub_stamp":"2024-04-27T07:04:14.566367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:39:04.238496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-27T07:04:17.494529+0000","last_change":"2024-04-27T07:04:14.626854+0000","last_active":"2024-04-27T07:04:17.494529+0000","last_peered":"2024-04-27T07:04:17.494529+0000","last_clean":"2024-04-27T07:04:17.494529+0000","last_became_active":"2024-04-27T07:04:14.626588+0000","last_became_peered":"2024-04-27T07:04:14.626588+0000","last_unstale":"2024-04-27T07:04:17.494529+0000","last_undegraded":"2024-04-27T07:04:17.494529+0000","last_fullsized":"2024-04-27T07:04:17.494529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:13.554072+0000","last_clean_scrub_stamp":"2024-04-27T07:04:13.554072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:21.930932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:04:28.788 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-27T07:04:28.788 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-27T07:04:28.788 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-27T07:04:28.788 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-27T07:04:28.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.933+0000 7fd21c2d8640 1 Processor -- start 2024-04-27T07:04:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.933+0000 7fd21c2d8640 1 -- start start 2024-04-27T07:04:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.933+0000 7fd21c2d8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd21406da10 0x7fd214143570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.933+0000 7fd21c2d8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2141448e0 con 0x7fd2140b4b70 2024-04-27T07:04:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.933+0000 7fd21c2d8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd214144b80 con 0x7fd2140b3db0 2024-04-27T07:04:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.933+0000 7fd21c2d8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fd214144ef0 con 0x7fd21406da10 2024-04-27T07:04:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.933+0000 7fd21a074640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd2140b4b70 0x7fd2140b4f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59664/0 (socket says 172.21.15.12:59664) 2024-04-27T07:04:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.933+0000 7fd21a074640 1 -- 172.21.15.12:0/252153095 learned_addr learned my addr 172.21.15.12:0/252153095 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.933+0000 7fd21a875640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd21406da10 0x7fd214143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.933+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2154776394 0 0) 0x7fd2141448e0 con 0x7fd2140b4b70 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd200003530 con 0x7fd2140b4b70 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3669727999 0 0) 0x7fd200003530 con 0x7fd2140b4b70 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd21406da10 msgr2=0x7fd214143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd21406da10 0x7fd214143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 >> v1:172.21.15.53:6789/0 conn(0x7fd2140b3db0 legacy=0x7fd21406d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2140b4420 con 0x7fd2140b4b70 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fd210001680 con 0x7fd2140b4b70 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 >> v1:172.21.15.12:6789/0 conn(0x7fd2140b4b70 legacy=0x7fd2140b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2000039f0 0x7fd200003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd21406da10 0x7fd200004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd200004d50 0x7fd2000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd20000bad0 con 0x7fd2000039f0 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd20000bc80 con 0x7fd200004d50 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fd20000be80 con 0x7fd21406da10 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21a074640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2000039f0 0x7fd200003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21a875640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd200004d50 0x7fd2000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219873640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd21406da10 0x7fd200004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21a074640 1 -- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd21406da10 msgr2=0x7fd200004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21a074640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd21406da10 0x7fd200004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21a074640 1 -- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd200004d50 msgr2=0x7fd2000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21a074640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd200004d50 0x7fd2000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21a074640 1 -- 172.21.15.12:0/252153095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd200003530 con 0x7fd2000039f0 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21a875640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd200004d50 0x7fd2000051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:28.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219873640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd21406da10 0x7fd200004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21a074640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2000039f0 0x7fd200003e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c210 tx=0x7fd21001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b6a9510d7219ae server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210007d40 con 0x7fd2000039f0 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2100a9070 con 0x7fd2000039f0 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd219072640 1 -- 172.21.15.12:0/252153095 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210005d40 con 0x7fd2000039f0 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21c2d8640 1 -- 172.21.15.12:0/252153095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2000039f0 msgr2=0x7fd200003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21c2d8640 1 --2- 172.21.15.12:0/252153095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2000039f0 0x7fd200003e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c210 tx=0x7fd21001cbe0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21a875640 1 -- 172.21.15.12:0/252153095 reap_dead start 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21c2d8640 1 -- 172.21.15.12:0/252153095 shutdown_connections 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21c2d8640 1 -- 172.21.15.12:0/252153095 >> 172.21.15.12:0/252153095 conn(0x7fd214067d10 msgr2=0x7fd214068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21c2d8640 1 -- 172.21.15.12:0/252153095 shutdown_connections 2024-04-27T07:04:28.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.937+0000 7fd21c2d8640 1 -- 172.21.15.12:0/252153095 wait complete. 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 Processor -- start 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 -- start start 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd214141d90 0x7fd2141e1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd214072d30 0x7fd21413ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2141e1e80 0x7fd21413d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd21414fae0 con 0x7fd214072d30 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd214144050 con 0x7fd2141e1e80 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fd21414fc60 con 0x7fd214141d90 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd219873640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd214072d30 0x7fd21413ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd219873640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd214072d30 0x7fd21413ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55664/0 (socket says 172.21.15.12:55664) 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd219873640 1 -- 172.21.15.12:0/3842926124 learned_addr learned my addr 172.21.15.12:0/3842926124 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21a074640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd214141d90 0x7fd2141e1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21a875640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2141e1e80 0x7fd21413d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd219873640 1 -- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd214141d90 msgr2=0x7fd2141e1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd219873640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd214141d90 0x7fd2141e1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd219873640 1 -- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2141e1e80 msgr2=0x7fd21413d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd219873640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2141e1e80 0x7fd21413d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd219873640 1 -- 172.21.15.12:0/3842926124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21413d8e0 con 0x7fd214072d30 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21a875640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2141e1e80 0x7fd21413d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd219873640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd214072d30 0x7fd21413ce60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd20400d7a0 tx=0x7fd20400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd20affd640 1 -- 172.21.15.12:0/3842926124 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204005270 con 0x7fd214072d30 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd20affd640 1 -- 172.21.15.12:0/3842926124 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd204017070 con 0x7fd214072d30 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 -- 172.21.15.12:0/3842926124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd214142170 con 0x7fd214072d30 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd20affd640 1 -- 172.21.15.12:0/3842926124 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20400b6b0 con 0x7fd214072d30 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 -- 172.21.15.12:0/3842926124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd214071660 con 0x7fd214072d30 2024-04-27T07:04:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd21c2d8640 1 -- 172.21.15.12:0/3842926124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd214068c20 con 0x7fd214072d30 2024-04-27T07:04:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd20affd640 1 -- 172.21.15.12:0/3842926124 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd20400b850 con 0x7fd214072d30 2024-04-27T07:04:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.941+0000 7fd20affd640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd1f8068380 0x7fd1f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.945+0000 7fd20affd640 1 -- 172.21.15.12:0/3842926124 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd204086820 con 0x7fd214072d30 2024-04-27T07:04:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.945+0000 7fd21a074640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd1f8068380 0x7fd1f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.945+0000 7fd21a074640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd1f8068380 0x7fd1f806a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd2100ab000 tx=0x7fd2100b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:28.949+0000 7fd20affd640 1 -- 172.21.15.12:0/3842926124 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd204056e70 con 0x7fd214072d30 2024-04-27T07:04:29.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.141+0000 7fd21c2d8640 1 -- 172.21.15.12:0/3842926124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd21409c930 con 0x7fd214072d30 2024-04-27T07:04:29.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.141+0000 7fd20affd640 1 -- 172.21.15.12:0/3842926124 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd204056770 con 0x7fd214072d30 2024-04-27T07:04:29.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:29.146 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-27T07:04:29.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.141+0000 7fd21c2d8640 1 -- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd1f8068380 msgr2=0x7fd1f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.141+0000 7fd21c2d8640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd1f8068380 0x7fd1f806a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd2100ab000 tx=0x7fd2100b0040 comp rx=0 tx=0).stop 2024-04-27T07:04:29.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.141+0000 7fd21c2d8640 1 -- 172.21.15.12:0/3842926124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd214072d30 msgr2=0x7fd21413ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.141+0000 7fd21c2d8640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd214072d30 0x7fd21413ce60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd20400d7a0 tx=0x7fd20400dc60 comp rx=0 tx=0).stop 2024-04-27T07:04:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.145+0000 7fd21c2d8640 1 -- 172.21.15.12:0/3842926124 shutdown_connections 2024-04-27T07:04:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.145+0000 7fd21c2d8640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd1f8068380 0x7fd1f806a840 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.145+0000 7fd21c2d8640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2141e1e80 0x7fd21413d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.145+0000 7fd21c2d8640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd214072d30 0x7fd21413ce60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.145+0000 7fd21c2d8640 1 --2- 172.21.15.12:0/3842926124 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd214141d90 0x7fd2141e1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.145+0000 7fd21c2d8640 1 -- 172.21.15.12:0/3842926124 >> 172.21.15.12:0/3842926124 conn(0x7fd214067d10 msgr2=0x7fd2140b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.145+0000 7fd21c2d8640 1 -- 172.21.15.12:0/3842926124 shutdown_connections 2024-04-27T07:04:29.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.145+0000 7fd21c2d8640 1 -- 172.21.15.12:0/3842926124 wait complete. 2024-04-27T07:04:29.163 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-27T07:04:29.164 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:29.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.269+0000 7fd0ae2f8640 1 Processor -- start 2024-04-27T07:04:29.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.269+0000 7fd0ae2f8640 1 -- start start 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ae2f8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd0a806ddc0 0x7fd0a8148520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ae2f8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd0a8148a60 0x7fd0a81525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ae2f8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a80b9df0 con 0x7fd0a80b9a10 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ae2f8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a80b4590 con 0x7fd0a8148a60 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ae2f8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a80740b0 con 0x7fd0a806ddc0 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ac895640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd0a8148a60 0x7fd0a81525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ac895640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd0a8148a60 0x7fd0a81525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55678/0 (socket says 172.21.15.12:55678) 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0a7fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd0a80b9a10 0x7fd0a80739a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59680/0 (socket says 172.21.15.12:59680) 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ac895640 1 -- 172.21.15.12:0/3149612413 learned_addr learned my addr 172.21.15.12:0/3149612413 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0a77fe640 1 --2- 172.21.15.12:0/3149612413 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd0a806ddc0 0x7fd0a8148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ac895640 1 -- 172.21.15.12:0/3149612413 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd0a806ddc0 msgr2=0x7fd0a8148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:29.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ac895640 1 --2- 172.21.15.12:0/3149612413 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd0a806ddc0 0x7fd0a8148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ac895640 1 -- 172.21.15.12:0/3149612413 >> v1:172.21.15.12:6789/0 conn(0x7fd0a80b9a10 legacy=0x7fd0a80739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ac895640 1 -- 172.21.15.12:0/3149612413 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a80b47e0 con 0x7fd0a8148a60 2024-04-27T07:04:29.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ac895640 1 --2- 172.21.15.12:0/3149612413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a8148a60 0x7fd0a81525b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009870 tx=0x7fd0a001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e560241600456dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:29.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/3149612413 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00a5040 con 0x7fd0a8148a60 2024-04-27T07:04:29.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/3149612413 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a00ae070 con 0x7fd0a8148a60 2024-04-27T07:04:29.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/3149612413 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00a9860 con 0x7fd0a8148a60 2024-04-27T07:04:29.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/3149612413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a8148a60 msgr2=0x7fd0a81525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.273+0000 7fd0ae2f8640 1 --2- 172.21.15.12:0/3149612413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a8148a60 0x7fd0a81525b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009870 tx=0x7fd0a001c040 comp rx=0 tx=0).stop 2024-04-27T07:04:29.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/3149612413 shutdown_connections 2024-04-27T07:04:29.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 --2- 172.21.15.12:0/3149612413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a8148a60 0x7fd0a81525b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 --2- 172.21.15.12:0/3149612413 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd0a806ddc0 0x7fd0a8148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/3149612413 >> 172.21.15.12:0/3149612413 conn(0x7fd0a8067d10 msgr2=0x7fd0a8068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:29.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/3149612413 shutdown_connections 2024-04-27T07:04:29.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/3149612413 wait complete. 2024-04-27T07:04:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 Processor -- start 2024-04-27T07:04:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- start start 2024-04-27T07:04:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a80b9a10 0x7fd0a81e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a8148a60 0x7fd0a8151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd0a8151ec0 0x7fd0a80b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8155ec0 con 0x7fd0a80b9a10 2024-04-27T07:04:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a8155d40 con 0x7fd0a8148a60 2024-04-27T07:04:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a8156040 con 0x7fd0a8151ec0 2024-04-27T07:04:29.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a80b9a10 0x7fd0a81e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a80b9a10 0x7fd0a81e4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55682/0 (socket says 172.21.15.12:55682) 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a7fff640 1 -- 172.21.15.12:0/1150764653 learned_addr learned my addr 172.21.15.12:0/1150764653 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ac895640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd0a8151ec0 0x7fd0a80b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a77fe640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a8148a60 0x7fd0a8151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a7fff640 1 -- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd0a8151ec0 msgr2=0x7fd0a80b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a7fff640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd0a8151ec0 0x7fd0a80b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a7fff640 1 -- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a8148a60 msgr2=0x7fd0a8151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a7fff640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a8148a60 0x7fd0a8151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a7fff640 1 -- 172.21.15.12:0/1150764653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a81e5670 con 0x7fd0a80b9a10 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a77fe640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a8148a60 0x7fd0a8151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a7fff640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a80b9a10 0x7fd0a81e4a10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd09800c7b0 tx=0x7fd09800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a4ff9640 1 -- 172.21.15.12:0/1150764653 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098018070 con 0x7fd0a80b9a10 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a4ff9640 1 -- 172.21.15.12:0/1150764653 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0980072d0 con 0x7fd0a80b9a10 2024-04-27T07:04:29.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/1150764653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a81491b0 con 0x7fd0a80b9a10 2024-04-27T07:04:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0a4ff9640 1 -- 172.21.15.12:0/1150764653 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980047f0 con 0x7fd0a80b9a10 2024-04-27T07:04:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/1150764653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a81e50b0 con 0x7fd0a80b9a10 2024-04-27T07:04:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.277+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/1150764653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a8068c20 con 0x7fd0a80b9a10 2024-04-27T07:04:29.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.281+0000 7fd0a4ff9640 1 -- 172.21.15.12:0/1150764653 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd09801b070 con 0x7fd0a80b9a10 2024-04-27T07:04:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.281+0000 7fd0a4ff9640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd094068380 0x7fd09406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.281+0000 7fd0a4ff9640 1 -- 172.21.15.12:0/1150764653 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd098085d90 con 0x7fd0a80b9a10 2024-04-27T07:04:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.281+0000 7fd0a77fe640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd094068380 0x7fd09406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.281+0000 7fd0a77fe640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd094068380 0x7fd09406a840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80b0f90 tx=0x7fd09c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:29.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.285+0000 7fd0a4ff9640 1 -- 172.21.15.12:0/1150764653 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd0980563e0 con 0x7fd0a80b9a10 2024-04-27T07:04:29.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.441+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/1150764653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0a809e4b0 con 0x7fd0a80b9a10 2024-04-27T07:04:29.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.441+0000 7fd0a4ff9640 1 -- 172.21.15.12:0/1150764653 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fd098055ce0 con 0x7fd0a80b9a10 2024-04-27T07:04:29.447 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:29.447 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:29.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd094068380 msgr2=0x7fd09406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd094068380 0x7fd09406a840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80b0f90 tx=0x7fd09c00b040 comp rx=0 tx=0).stop 2024-04-27T07:04:29.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/1150764653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a80b9a10 msgr2=0x7fd0a81e4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a80b9a10 0x7fd0a81e4a10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd09800c7b0 tx=0x7fd09800cc70 comp rx=0 tx=0).stop 2024-04-27T07:04:29.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/1150764653 shutdown_connections 2024-04-27T07:04:29.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd094068380 0x7fd09406a840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd0a8151ec0 0x7fd0a80b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a8148a60 0x7fd0a8151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 --2- 172.21.15.12:0/1150764653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0a80b9a10 0x7fd0a81e4a10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/1150764653 >> 172.21.15.12:0/1150764653 conn(0x7fd0a8067d10 msgr2=0x7fd0a809b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:29.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/1150764653 shutdown_connections 2024-04-27T07:04:29.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.445+0000 7fd0ae2f8640 1 -- 172.21.15.12:0/1150764653 wait complete. 2024-04-27T07:04:29.465 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:12.742740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:13.912898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:29.466 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:29.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ef1e640 1 Processor -- start 2024-04-27T07:04:29.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ef1e640 1 -- start start 2024-04-27T07:04:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ef1e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581489c0 0x7f22580b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ef1e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22580b6860 con 0x7f2258142010 2024-04-27T07:04:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ef1e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2258147ee0 con 0x7f22580b6480 2024-04-27T07:04:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ef1e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f22581481c0 con 0x7f22581489c0 2024-04-27T07:04:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225d4bb640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2258142010 0x7f2258152650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59692/0 (socket says 172.21.15.12:59692) 2024-04-27T07:04:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225d4bb640 1 -- 172.21.15.12:0/2677231203 learned_addr learned my addr 172.21.15.12:0/2677231203 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f2257fff640 1 --2- 172.21.15.12:0/2677231203 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581489c0 0x7f22580b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 408550072 0 0) 0x7f22580b6860 con 0x7f2258142010 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2240003530 con 0x7f2258142010 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1646355476 0 0) 0x7f2240003530 con 0x7f2258142010 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581489c0 msgr2=0x7f22580b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 --2- 172.21.15.12:0/2677231203 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581489c0 0x7f22580b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 >> v1:172.21.15.53:6789/0 conn(0x7f22580b6480 legacy=0x7f2258141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22581484a0 con 0x7f2258142010 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f2250001a00 con 0x7f2258142010 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 >> v1:172.21.15.12:6789/0 conn(0x7f2258142010 legacy=0x7f2258152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 --2- 172.21.15.12:0/2677231203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 0x7f2240003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 --2- 172.21.15.12:0/2677231203 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581489c0 0x7f2240004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 --2- 172.21.15.12:0/2677231203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2240004d50 0x7f22400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f22581481c0 con 0x7f22400039f0 2024-04-27T07:04:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f224000ba80 con 0x7f2240004d50 2024-04-27T07:04:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f224000bc80 con 0x7f22581489c0 2024-04-27T07:04:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ccba640 1 --2- 172.21.15.12:0/2677231203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 0x7f2240003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ccba640 1 -- 172.21.15.12:0/2677231203 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581489c0 msgr2=0x7f2240004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ccba640 1 --2- 172.21.15.12:0/2677231203 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581489c0 0x7f2240004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ccba640 1 -- 172.21.15.12:0/2677231203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2240004d50 msgr2=0x7f22400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ccba640 1 --2- 172.21.15.12:0/2677231203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2240004d50 0x7f22400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.585+0000 7f225ccba640 1 -- 172.21.15.12:0/2677231203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2240003530 con 0x7f22400039f0 2024-04-27T07:04:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ccba640 1 --2- 172.21.15.12:0/2677231203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 0x7f2240003dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f224c001910 tx=0x7f224c01c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b6accdcf3ec420 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c0b0070 con 0x7f22400039f0 2024-04-27T07:04:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224c0ab620 con 0x7f22400039f0 2024-04-27T07:04:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f22577fe640 1 -- 172.21.15.12:0/2677231203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c001510 con 0x7f22400039f0 2024-04-27T07:04:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 -- 172.21.15.12:0/2677231203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 msgr2=0x7f2240003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 --2- 172.21.15.12:0/2677231203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 0x7f2240003dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f224c001910 tx=0x7f224c01c580 comp rx=0 tx=0).stop 2024-04-27T07:04:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225d4bb640 1 -- 172.21.15.12:0/2677231203 reap_dead start 2024-04-27T07:04:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 -- 172.21.15.12:0/2677231203 shutdown_connections 2024-04-27T07:04:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 -- 172.21.15.12:0/2677231203 >> 172.21.15.12:0/2677231203 conn(0x7f22580bd620 msgr2=0x7f22580bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 -- 172.21.15.12:0/2677231203 shutdown_connections 2024-04-27T07:04:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 -- 172.21.15.12:0/2677231203 wait complete. 2024-04-27T07:04:29.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 Processor -- start 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 -- start start 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 0x7f2258152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22581e4820 0x7f22581e4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581389a0 0x7f2258138e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2258154580 con 0x7f22400039f0 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2258142440 con 0x7f22581e4820 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ef1e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f2258154700 con 0x7f22581389a0 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ccba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 0x7f2258152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ccba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 0x7f2258152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55698/0 (socket says 172.21.15.12:55698) 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.589+0000 7f225ccba640 1 -- 172.21.15.12:0/477172515 learned_addr learned my addr 172.21.15.12:0/477172515 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225d4bb640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581389a0 0x7f2258138e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f2257fff640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22581e4820 0x7f22581e4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225ccba640 1 -- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581389a0 msgr2=0x7f2258138e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225ccba640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581389a0 0x7f2258138e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225ccba640 1 -- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22581e4820 msgr2=0x7f22581e4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225ccba640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22581e4820 0x7f22581e4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225ccba640 1 -- 172.21.15.12:0/477172515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2258151330 con 0x7f22400039f0 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225d4bb640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581389a0 0x7f2258138e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f2257fff640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22581e4820 0x7f22581e4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225ccba640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 0x7f2258152720 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f224c0ae040 tx=0x7f2240005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f22557fa640 1 -- 172.21.15.12:0/477172515 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c0bb020 con 0x7f22400039f0 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f22557fa640 1 -- 172.21.15.12:0/477172515 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224c0ab400 con 0x7f22400039f0 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225ef1e640 1 -- 172.21.15.12:0/477172515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225813b5c0 con 0x7f22400039f0 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f22557fa640 1 -- 172.21.15.12:0/477172515 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c0b0040 con 0x7f22400039f0 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225ef1e640 1 -- 172.21.15.12:0/477172515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2258151bd0 con 0x7f22400039f0 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f225ef1e640 1 -- 172.21.15.12:0/477172515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2258068c20 con 0x7f22400039f0 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f22557fa640 1 -- 172.21.15.12:0/477172515 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f224c0be020 con 0x7f22400039f0 2024-04-27T07:04:29.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f22557fa640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22440683b0 0x7f224406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f22557fa640 1 -- 172.21.15.12:0/477172515 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f224c120ec0 con 0x7f22400039f0 2024-04-27T07:04:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f2257fff640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22440683b0 0x7f224406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.593+0000 7f2257fff640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22440683b0 0x7f224406a870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f22581418d0 tx=0x7f2248008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:29.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.597+0000 7f22557fa640 1 -- 172.21.15.12:0/477172515 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f224c0c5030 con 0x7f22400039f0 2024-04-27T07:04:29.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.761+0000 7f225ef1e640 1 -- 172.21.15.12:0/477172515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f22580bc8f0 con 0x7f22400039f0 2024-04-27T07:04:29.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.761+0000 7f22557fa640 1 -- 172.21.15.12:0/477172515 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f224c016290 con 0x7f22400039f0 2024-04-27T07:04:29.767 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 -- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22440683b0 msgr2=0x7f224406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22440683b0 0x7f224406a870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f22581418d0 tx=0x7f2248008040 comp rx=0 tx=0).stop 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 -- 172.21.15.12:0/477172515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 msgr2=0x7f2258152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 0x7f2258152720 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f224c0ae040 tx=0x7f2240005840 comp rx=0 tx=0).stop 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 -- 172.21.15.12:0/477172515 shutdown_connections 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f22440683b0 0x7f224406a870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f22581389a0 0x7f2258138e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f22581e4820 0x7f22581e4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 --2- 172.21.15.12:0/477172515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22400039f0 0x7f2258152720 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 -- 172.21.15.12:0/477172515 >> 172.21.15.12:0/477172515 conn(0x7f22580bd620 msgr2=0x7f22580bbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 -- 172.21.15.12:0/477172515 shutdown_connections 2024-04-27T07:04:29.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.765+0000 7f225ef1e640 1 -- 172.21.15.12:0/477172515 wait complete. 2024-04-27T07:04:29.787 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:04:29.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe694c96640 1 Processor -- start 2024-04-27T07:04:29.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe694c96640 1 -- start start 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe694c96640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6900b11b0 0x7fe69013fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe694c96640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe69013ffb0 0x7fe6901503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe694c96640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6901460c0 con 0x7fe6900731b0 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe694c96640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe6900b0b90 con 0x7fe69013ffb0 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe694c96640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe6901463c0 con 0x7fe6900b11b0 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68e59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe6900731b0 0x7fe690149030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59694/0 (socket says 172.21.15.12:59694) 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68e59c640 1 -- 172.21.15.12:0/4212350943 learned_addr learned my addr 172.21.15.12:0/4212350943 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68dd9b640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6900b11b0 0x7fe69013fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 688169428 0 0) 0x7fe6901460c0 con 0x7fe6900731b0 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe67c003530 con 0x7fe6900731b0 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68ed9d640 1 --2- 172.21.15.12:0/4212350943 >> v2:172.21.15.53:3300/0 conn(0x7fe69013ffb0 0x7fe6901503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2916862154 0 0) 0x7fe67c003530 con 0x7fe6900731b0 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6900b11b0 msgr2=0x7fe69013fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6900b11b0 0x7fe69013fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 >> v2:172.21.15.53:3300/0 conn(0x7fe69013ffb0 msgr2=0x7fe6901503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 --2- 172.21.15.12:0/4212350943 >> v2:172.21.15.53:3300/0 conn(0x7fe69013ffb0 0x7fe6901503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6900b4800 con 0x7fe6900731b0 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fe688001a00 con 0x7fe6900731b0 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68ed9d640 1 --2- 172.21.15.12:0/4212350943 >> v2:172.21.15.53:3300/0 conn(0x7fe69013ffb0 0x7fe6901503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 >> v1:172.21.15.12:6789/0 conn(0x7fe6900731b0 legacy=0x7fe690149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe67c0039c0 0x7fe67c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe67c004810 0x7fe67c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6900b11b0 0x7fe67c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe6900b0b90 con 0x7fe67c0039c0 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901463c0 con 0x7fe67c004810 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe67c00bb00 con 0x7fe6900b11b0 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68e59c640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe67c0039c0 0x7fe67c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68ed9d640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6900b11b0 0x7fe67c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68dd9b640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe67c004810 0x7fe67c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68e59c640 1 -- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6900b11b0 msgr2=0x7fe67c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:29.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68e59c640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe6900b11b0 0x7fe67c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68e59c640 1 -- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe67c004810 msgr2=0x7fe67c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:29.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68e59c640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe67c004810 0x7fe67c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.913+0000 7fe68e59c640 1 -- 172.21.15.12:0/4212350943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67c003530 con 0x7fe67c0039c0 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe68e59c640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe67c0039c0 0x7fe67c003de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe68800a870 tx=0x7fe68801d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d50e3fe3e23803 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68800a4c0 con 0x7fe67c0039c0 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6880ad630 con 0x7fe67c0039c0 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe68d59a640 1 -- 172.21.15.12:0/4212350943 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6880ada20 con 0x7fe67c0039c0 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 -- 172.21.15.12:0/4212350943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe67c0039c0 msgr2=0x7fe67c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 --2- 172.21.15.12:0/4212350943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe67c0039c0 0x7fe67c003de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe68800a870 tx=0x7fe68801d3a0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe68ed9d640 1 -- 172.21.15.12:0/4212350943 reap_dead start 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 -- 172.21.15.12:0/4212350943 shutdown_connections 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 -- 172.21.15.12:0/4212350943 >> 172.21.15.12:0/4212350943 conn(0x7fe6900b9e80 msgr2=0x7fe6900ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 -- 172.21.15.12:0/4212350943 shutdown_connections 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 -- 172.21.15.12:0/4212350943 wait complete. 2024-04-27T07:04:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 Processor -- start 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 -- start start 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe67c0039c0 0x7fe69013a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6900b7ab0 0x7fe6901e1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6901e2500 0x7fe6900b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe690151110 con 0x7fe6900b7ab0 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe690146a60 con 0x7fe6901e2500 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.917+0000 7fe694c96640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe690151290 con 0x7fe67c0039c0 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6900b7ab0 0x7fe6901e1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6901e2500 0x7fe6900b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6900b7ab0 0x7fe6901e1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55706/0 (socket says 172.21.15.12:55706) 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68dd9b640 1 -- 172.21.15.12:0/370244437 learned_addr learned my addr 172.21.15.12:0/370244437 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68e59c640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe67c0039c0 0x7fe69013a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68dd9b640 1 -- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe67c0039c0 msgr2=0x7fe69013a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68dd9b640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe67c0039c0 0x7fe69013a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68dd9b640 1 -- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6901e2500 msgr2=0x7fe6900b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68dd9b640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6901e2500 0x7fe6900b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68dd9b640 1 -- 172.21.15.12:0/370244437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6901e2990 con 0x7fe6900b7ab0 2024-04-27T07:04:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68dd9b640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6900b7ab0 0x7fe6901e1fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe678010810 tx=0x7fe678010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe686ffd640 1 -- 172.21.15.12:0/370244437 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67800f040 con 0x7fe6900b7ab0 2024-04-27T07:04:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe686ffd640 1 -- 172.21.15.12:0/370244437 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67800eb60 con 0x7fe6900b7ab0 2024-04-27T07:04:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe686ffd640 1 -- 172.21.15.12:0/370244437 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67801c400 con 0x7fe6900b7ab0 2024-04-27T07:04:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe694c96640 1 -- 172.21.15.12:0/370244437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6900b7f90 con 0x7fe6900b7ab0 2024-04-27T07:04:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe694c96640 1 -- 172.21.15.12:0/370244437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6900b96c0 con 0x7fe6900b7ab0 2024-04-27T07:04:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe694c96640 1 -- 172.21.15.12:0/370244437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe690069330 con 0x7fe6900b7ab0 2024-04-27T07:04:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe686ffd640 1 -- 172.21.15.12:0/370244437 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe67801c5a0 con 0x7fe6900b7ab0 2024-04-27T07:04:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe686ffd640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6700683b0 0x7fe67006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe686ffd640 1 -- 172.21.15.12:0/370244437 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe678086dc0 con 0x7fe6900b7ab0 2024-04-27T07:04:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.921+0000 7fe68e59c640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6700683b0 0x7fe67006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.925+0000 7fe68e59c640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6700683b0 0x7fe67006a870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6880ad000 tx=0x7fe688001370 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:29.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:29.925+0000 7fe686ffd640 1 -- 172.21.15.12:0/370244437 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe678018030 con 0x7fe6900b7ab0 2024-04-27T07:04:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.093+0000 7fe694c96640 1 -- 172.21.15.12:0/370244437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe69014c180 con 0x7fe6900b7ab0 2024-04-27T07:04:30.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.093+0000 7fe686ffd640 1 -- 172.21.15.12:0/370244437 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fe678056dc0 con 0x7fe6900b7ab0 2024-04-27T07:04:30.098 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:04:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.093+0000 7fe694c96640 1 -- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6700683b0 msgr2=0x7fe67006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.093+0000 7fe694c96640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6700683b0 0x7fe67006a870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6880ad000 tx=0x7fe688001370 comp rx=0 tx=0).stop 2024-04-27T07:04:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.097+0000 7fe694c96640 1 -- 172.21.15.12:0/370244437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6900b7ab0 msgr2=0x7fe6901e1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.097+0000 7fe694c96640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6900b7ab0 0x7fe6901e1fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe678010810 tx=0x7fe678010cd0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.097+0000 7fe694c96640 1 -- 172.21.15.12:0/370244437 shutdown_connections 2024-04-27T07:04:30.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.097+0000 7fe694c96640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe6700683b0 0x7fe67006a870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.097+0000 7fe694c96640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6901e2500 0x7fe6900b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.097+0000 7fe694c96640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6900b7ab0 0x7fe6901e1fc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.097+0000 7fe694c96640 1 --2- 172.21.15.12:0/370244437 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe67c0039c0 0x7fe69013a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.097+0000 7fe694c96640 1 -- 172.21.15.12:0/370244437 >> 172.21.15.12:0/370244437 conn(0x7fe6900b9e80 msgr2=0x7fe69013e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:30.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.097+0000 7fe694c96640 1 -- 172.21.15.12:0/370244437 shutdown_connections 2024-04-27T07:04:30.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.097+0000 7fe694c96640 1 -- 172.21.15.12:0/370244437 wait complete. 2024-04-27T07:04:30.117 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:04:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53abde4640 1 Processor -- start 2024-04-27T07:04:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53abde4640 1 -- start start 2024-04-27T07:04:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53abde4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f53a40b5620 0x7f53a41490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53abde4640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a40b7870 con 0x7f53a40ae8f0 2024-04-27T07:04:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53abde4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a40b4860 con 0x7f53a40b4480 2024-04-27T07:04:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53abde4640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f53a40b4c40 con 0x7f53a40b5620 2024-04-27T07:04:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53a937f640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f53a40ae8f0 0x7f53a414e3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59706/0 (socket says 172.21.15.12:59706) 2024-04-27T07:04:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53a937f640 1 -- 172.21.15.12:0/4218381722 learned_addr learned my addr 172.21.15.12:0/4218381722 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53aa381640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f53a40b5620 0x7f53a41490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53a8b7e640 1 -- 172.21.15.12:0/4218381722 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2377109317 0 0) 0x7f53a40b7870 con 0x7f53a40ae8f0 2024-04-27T07:04:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53a8b7e640 1 -- 172.21.15.12:0/4218381722 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5390003530 con 0x7f53a40ae8f0 2024-04-27T07:04:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53aa381640 1 -- 172.21.15.12:0/4218381722 >> v1:172.21.15.53:6789/0 conn(0x7f53a40b4480 legacy=0x7f53a4073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53aa381640 1 -- 172.21.15.12:0/4218381722 >> v1:172.21.15.12:6789/0 conn(0x7f53a40ae8f0 legacy=0x7f53a414e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.241+0000 7f53aa381640 1 -- 172.21.15.12:0/4218381722 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a4074020 con 0x7f53a40b5620 2024-04-27T07:04:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53aa381640 1 --2- 172.21.15.12:0/4218381722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b5620 0x7f53a41490b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f539c009d90 tx=0x7f539c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1a06b47cf80ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a8b7e640 1 -- 172.21.15.12:0/4218381722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c0b2070 con 0x7f53a40b5620 2024-04-27T07:04:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a8b7e640 1 -- 172.21.15.12:0/4218381722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f539c0a5710 con 0x7f53a40b5620 2024-04-27T07:04:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a8b7e640 1 -- 172.21.15.12:0/4218381722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c0a5ac0 con 0x7f53a40b5620 2024-04-27T07:04:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 -- 172.21.15.12:0/4218381722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b5620 msgr2=0x7f53a41490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 --2- 172.21.15.12:0/4218381722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b5620 0x7f53a41490b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f539c009d90 tx=0x7f539c01c850 comp rx=0 tx=0).stop 2024-04-27T07:04:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 -- 172.21.15.12:0/4218381722 shutdown_connections 2024-04-27T07:04:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 --2- 172.21.15.12:0/4218381722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b5620 0x7f53a41490b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 -- 172.21.15.12:0/4218381722 >> 172.21.15.12:0/4218381722 conn(0x7f53a40bb4f0 msgr2=0x7f53a40bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 -- 172.21.15.12:0/4218381722 shutdown_connections 2024-04-27T07:04:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 -- 172.21.15.12:0/4218381722 wait complete. 2024-04-27T07:04:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 Processor -- start 2024-04-27T07:04:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 -- start start 2024-04-27T07:04:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53a40ae8f0 0x7f53a40b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b4480 0x7f53a4136b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f53a40b5620 0x7f53a4135540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f53a40b2c00 con 0x7f53a40b4480 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f53a40b7df0 con 0x7f53a40ae8f0 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53abde4640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f53a40b1b30 con 0x7f53a40b5620 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a937f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b4480 0x7f53a4136b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a937f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b4480 0x7f53a4136b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55714/0 (socket says 172.21.15.12:55714) 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a937f640 1 -- 172.21.15.12:0/3544547521 learned_addr learned my addr 172.21.15.12:0/3544547521 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a9b80640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53a40ae8f0 0x7f53a40b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53aa381640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f53a40b5620 0x7f53a4135540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a937f640 1 -- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f53a40b5620 msgr2=0x7f53a4135540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a937f640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f53a40b5620 0x7f53a4135540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a937f640 1 -- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53a40ae8f0 msgr2=0x7f53a40b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a937f640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53a40ae8f0 0x7f53a40b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.245+0000 7f53a937f640 1 -- 172.21.15.12:0/3544547521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a40b1200 con 0x7f53a40b4480 2024-04-27T07:04:30.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f53a937f640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b4480 0x7f53a4136b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f539400c500 tx=0x7f539400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:30.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f539a7fc640 1 -- 172.21.15.12:0/3544547521 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394017070 con 0x7f53a40b4480 2024-04-27T07:04:30.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f539a7fc640 1 -- 172.21.15.12:0/3544547521 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5394005cf0 con 0x7f53a40b4480 2024-04-27T07:04:30.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f539a7fc640 1 -- 172.21.15.12:0/3544547521 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940072f0 con 0x7f53a40b4480 2024-04-27T07:04:30.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f53abde4640 1 -- 172.21.15.12:0/3544547521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a40b14e0 con 0x7f53a40b4480 2024-04-27T07:04:30.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f53abde4640 1 -- 172.21.15.12:0/3544547521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a40b1fc0 con 0x7f53a40b4480 2024-04-27T07:04:30.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f53abde4640 1 -- 172.21.15.12:0/3544547521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a4068c20 con 0x7f53a40b4480 2024-04-27T07:04:30.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f539a7fc640 1 -- 172.21.15.12:0/3544547521 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5394007490 con 0x7f53a40b4480 2024-04-27T07:04:30.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f539a7fc640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5388068380 0x7f538806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f539a7fc640 1 -- 172.21.15.12:0/3544547521 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5394085c70 con 0x7f53a40b4480 2024-04-27T07:04:30.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f53a9b80640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5388068380 0x7f538806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.249+0000 7f53a9b80640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5388068380 0x7f538806a840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f53a0007fd0 tx=0x7f53a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:30.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.253+0000 7f539a7fc640 1 -- 172.21.15.12:0/3544547521 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5394055d30 con 0x7f53a40b4480 2024-04-27T07:04:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.417+0000 7f53abde4640 1 -- 172.21.15.12:0/3544547521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f53a4151330 con 0x7f53a40b4480 2024-04-27T07:04:30.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.417+0000 7f539a7fc640 1 -- 172.21.15.12:0/3544547521 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f539405a030 con 0x7f53a40b4480 2024-04-27T07:04:30.424 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:04:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 -- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5388068380 msgr2=0x7f538806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5388068380 0x7f538806a840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f53a0007fd0 tx=0x7f53a0009040 comp rx=0 tx=0).stop 2024-04-27T07:04:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 -- 172.21.15.12:0/3544547521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b4480 msgr2=0x7f53a4136b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b4480 0x7f53a4136b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f539400c500 tx=0x7f539400c9c0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 -- 172.21.15.12:0/3544547521 shutdown_connections 2024-04-27T07:04:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5388068380 0x7f538806a840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f53a40b5620 0x7f53a4135540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53a40b4480 0x7f53a4136b80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 --2- 172.21.15.12:0/3544547521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53a40ae8f0 0x7f53a40b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 -- 172.21.15.12:0/3544547521 >> 172.21.15.12:0/3544547521 conn(0x7f53a40bb4f0 msgr2=0x7f53a40b86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:30.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 -- 172.21.15.12:0/3544547521 shutdown_connections 2024-04-27T07:04:30.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.421+0000 7f53abde4640 1 -- 172.21.15.12:0/3544547521 wait complete. 2024-04-27T07:04:30.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:04:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1cb24b640 1 Processor -- start 2024-04-27T07:04:30.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1cb24b640 1 -- start start 2024-04-27T07:04:30.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1cb24b640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c4148310 0x7fc1c414b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1cb24b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c40b7990 con 0x7fc1c4141620 2024-04-27T07:04:30.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1cb24b640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c40bade0 con 0x7fc1c40baa00 2024-04-27T07:04:30.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1cb24b640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c40b73d0 con 0x7fc1c4148310 2024-04-27T07:04:30.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c97e8640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc1c4141620 0x7fc1c4141ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59722/0 (socket says 172.21.15.12:59722) 2024-04-27T07:04:30.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c97e8640 1 -- 172.21.15.12:0/2532360798 learned_addr learned my addr 172.21.15.12:0/2532360798 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:30.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c3fff640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c4148310 0x7fc1c414b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 94371130 0 0) 0x7fc1bc0014a0 con 0x7fc1c4141620 2024-04-27T07:04:30.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1ac003530 con 0x7fc1c4141620 2024-04-27T07:04:30.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4220290664 0 0) 0x7fc1ac003530 con 0x7fc1c4141620 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c4148310 msgr2=0x7fc1c414b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c4148310 0x7fc1c414b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 >> v1:172.21.15.53:6789/0 conn(0x7fc1c40baa00 legacy=0x7fc1c4147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c40b76b0 con 0x7fc1c4141620 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fc1bc001bd0 con 0x7fc1c4141620 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 >> v1:172.21.15.12:6789/0 conn(0x7fc1c4141620 legacy=0x7fc1c4141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1ac0039f0 0x7fc1ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c4148310 0x7fc1ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac004d50 0x7fc1ac0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40b73d0 con 0x7fc1ac004d50 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac00bc10 con 0x7fc1ac0039f0 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ac00be10 con 0x7fc1c4148310 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c97e8640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac004d50 0x7fc1ac0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c3fff640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c4148310 0x7fc1ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c8fe7640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1ac0039f0 0x7fc1ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c97e8640 1 -- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c4148310 msgr2=0x7fc1ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:30.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c97e8640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c4148310 0x7fc1ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c97e8640 1 -- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1ac0039f0 msgr2=0x7fc1ac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:30.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c97e8640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1ac0039f0 0x7fc1ac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.561+0000 7fc1c97e8640 1 -- 172.21.15.12:0/2532360798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac003530 con 0x7fc1ac004d50 2024-04-27T07:04:30.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1c97e8640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac004d50 0x7fc1ac0051b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00a960 tx=0x7fc1bc01d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4416274304b2b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:30.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc00a580 con 0x7fc1ac004d50 2024-04-27T07:04:30.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc0ad630 con 0x7fc1ac004d50 2024-04-27T07:04:30.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1c37fe640 1 -- 172.21.15.12:0/2532360798 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0ada60 con 0x7fc1ac004d50 2024-04-27T07:04:30.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1cb24b640 1 -- 172.21.15.12:0/2532360798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac004d50 msgr2=0x7fc1ac0051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1cb24b640 1 --2- 172.21.15.12:0/2532360798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1ac004d50 0x7fc1ac0051b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00a960 tx=0x7fc1bc01d270 comp rx=0 tx=0).stop 2024-04-27T07:04:30.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1c97e8640 1 -- 172.21.15.12:0/2532360798 reap_dead start 2024-04-27T07:04:30.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1cb24b640 1 -- 172.21.15.12:0/2532360798 shutdown_connections 2024-04-27T07:04:30.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1cb24b640 1 -- 172.21.15.12:0/2532360798 >> 172.21.15.12:0/2532360798 conn(0x7fc1c40738e0 msgr2=0x7fc1c40ba090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:30.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1cb24b640 1 -- 172.21.15.12:0/2532360798 shutdown_connections 2024-04-27T07:04:30.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.565+0000 7fc1cb24b640 1 -- 172.21.15.12:0/2532360798 wait complete. 2024-04-27T07:04:30.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 Processor -- start 2024-04-27T07:04:30.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 -- start start 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1c41e3910 0x7fc1c4149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c413c7f0 0x7fc1c413cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c413d5c0 0x7fc1c413da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4153f20 con 0x7fc1c413d5c0 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4153da0 con 0x7fc1c41e3910 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c41540a0 con 0x7fc1c413c7f0 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c97e8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c413d5c0 0x7fc1c413da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c97e8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c413d5c0 0x7fc1c413da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55720/0 (socket says 172.21.15.12:55720) 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c97e8640 1 -- 172.21.15.12:0/3001375124 learned_addr learned my addr 172.21.15.12:0/3001375124 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c8fe7640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1c41e3910 0x7fc1c4149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c3fff640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c413c7f0 0x7fc1c413cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c97e8640 1 -- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c413c7f0 msgr2=0x7fc1c413cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c97e8640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c413c7f0 0x7fc1c413cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c97e8640 1 -- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1c41e3910 msgr2=0x7fc1c4149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c97e8640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1c41e3910 0x7fc1c4149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c97e8640 1 -- 172.21.15.12:0/3001375124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c4148dc0 con 0x7fc1c413d5c0 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c3fff640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c413c7f0 0x7fc1c413cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c97e8640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c413d5c0 0x7fc1c413da40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00a960 tx=0x7fc1ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c17fa640 1 -- 172.21.15.12:0/3001375124 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0bb070 con 0x7fc1c413d5c0 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c17fa640 1 -- 172.21.15.12:0/3001375124 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc0ad430 con 0x7fc1c413d5c0 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 -- 172.21.15.12:0/3001375124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c413d1b0 con 0x7fc1c413d5c0 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c17fa640 1 -- 172.21.15.12:0/3001375124 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc001b10 con 0x7fc1c413d5c0 2024-04-27T07:04:30.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 -- 172.21.15.12:0/3001375124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c414a4a0 con 0x7fc1c413d5c0 2024-04-27T07:04:30.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1cb24b640 1 -- 172.21.15.12:0/3001375124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c4148f90 con 0x7fc1c413d5c0 2024-04-27T07:04:30.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c17fa640 1 -- 172.21.15.12:0/3001375124 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc1bc0be070 con 0x7fc1c413d5c0 2024-04-27T07:04:30.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c17fa640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc1b00683b0 0x7fc1b006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c17fa640 1 -- 172.21.15.12:0/3001375124 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc1bc1205a0 con 0x7fc1c413d5c0 2024-04-27T07:04:30.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.569+0000 7fc1c8fe7640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc1b00683b0 0x7fc1b006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.573+0000 7fc1c8fe7640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc1b00683b0 0x7fc1b006a870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4000c10 tx=0x7fc1b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:30.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.573+0000 7fc1c17fa640 1 -- 172.21.15.12:0/3001375124 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc1c40b7990 con 0x7fc1c413d5c0 2024-04-27T07:04:30.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.777+0000 7fc1cb24b640 1 -- 172.21.15.12:0/3001375124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1c4068c20 con 0x7fc1c413d5c0 2024-04-27T07:04:30.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.777+0000 7fc1c17fa640 1 -- 172.21.15.12:0/3001375124 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3527 (secure 0 0 0) 0x7fc1bc0f0bf0 con 0x7fc1c413d5c0 2024-04-27T07:04:30.783 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:30.783 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1908224930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1908224930},{"type":"v1","addr":"172.21.15.53:6835","nonce":1908224930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:04:14.961460+0000","modified":"2024-04-27T07:04:17.982838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4242":{"gid":4242,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4173401303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4173401303},{"type":"v1","addr":"172.21.15.12:6833","nonce":4173401303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4242,"qdb_cluster":[4242]},"id":1}]} 2024-04-27T07:04:30.783 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-04-27T07:04:30.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 -- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc1b00683b0 msgr2=0x7fc1b006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc1b00683b0 0x7fc1b006a870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4000c10 tx=0x7fc1b4009040 comp rx=0 tx=0).stop 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 -- 172.21.15.12:0/3001375124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c413d5c0 msgr2=0x7fc1c413da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c413d5c0 0x7fc1c413da40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00a960 tx=0x7fc1ac005840 comp rx=0 tx=0).stop 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 -- 172.21.15.12:0/3001375124 shutdown_connections 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc1b00683b0 0x7fc1b006a870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1c413d5c0 0x7fc1c413da40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc1c413c7f0 0x7fc1c413cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 --2- 172.21.15.12:0/3001375124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1c41e3910 0x7fc1c4149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 -- 172.21.15.12:0/3001375124 >> 172.21.15.12:0/3001375124 conn(0x7fc1c40738e0 msgr2=0x7fc1c4145b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 -- 172.21.15.12:0/3001375124 shutdown_connections 2024-04-27T07:04:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.781+0000 7fc1cb24b640 1 -- 172.21.15.12:0/3001375124 wait complete. 2024-04-27T07:04:30.803 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:30.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.913+0000 7fbdbac96640 1 Processor -- start 2024-04-27T07:04:30.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.913+0000 7fbdbac96640 1 -- start start 2024-04-27T07:04:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdbac96640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbdb41487b0 0x7fbdb4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdbac96640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fbdb413e5b0 0x7fbdb413ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdbac96640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb40b9ec0 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdbac96640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb409ddb0 con 0x7fbdb41487b0 2024-04-27T07:04:30.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdbac96640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb409e090 con 0x7fbdb413e5b0 2024-04-27T07:04:30.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb3fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbdb41487b0 0x7fbdb4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb8a32640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbdb40b9ae0 0x7fbdb41480a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59738/0 (socket says 172.21.15.12:59738) 2024-04-27T07:04:30.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb8a32640 1 -- 172.21.15.12:0/359085301 learned_addr learned my addr 172.21.15.12:0/359085301 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:30.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb9233640 1 --2- 172.21.15.12:0/359085301 >> v2:172.21.15.53:3300/0 conn(0x7fbdb413e5b0 0x7fbdb413ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb3fff640 1 -- 172.21.15.12:0/359085301 >> v2:172.21.15.53:3300/0 conn(0x7fbdb413e5b0 msgr2=0x7fbdb413ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:30.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb3fff640 1 --2- 172.21.15.12:0/359085301 >> v2:172.21.15.53:3300/0 conn(0x7fbdb413e5b0 0x7fbdb413ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb3fff640 1 -- 172.21.15.12:0/359085301 >> v1:172.21.15.12:6789/0 conn(0x7fbdb40b9ae0 legacy=0x7fbdb41480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb3fff640 1 -- 172.21.15.12:0/359085301 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb413efa0 con 0x7fbdb41487b0 2024-04-27T07:04:30.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb37fe640 1 -- 172.21.15.12:0/359085301 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1985735691 0 0) 0x7fbdb40b9ec0 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb3fff640 1 --2- 172.21.15.12:0/359085301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41487b0 0x7fbdb4073c60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbda80097c0 tx=0x7fbda801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127f5c9f05367796 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:30.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb37fe640 1 -- 172.21.15.12:0/359085301 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80b0070 con 0x7fbdb41487b0 2024-04-27T07:04:30.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb37fe640 1 -- 172.21.15.12:0/359085301 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda80ab600 con 0x7fbdb41487b0 2024-04-27T07:04:30.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.917+0000 7fbdb37fe640 1 -- 172.21.15.12:0/359085301 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda80ab9d0 con 0x7fbdb41487b0 2024-04-27T07:04:30.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- 172.21.15.12:0/359085301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41487b0 msgr2=0x7fbdb4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 --2- 172.21.15.12:0/359085301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41487b0 0x7fbdb4073c60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbda80097c0 tx=0x7fbda801c460 comp rx=0 tx=0).stop 2024-04-27T07:04:30.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- 172.21.15.12:0/359085301 shutdown_connections 2024-04-27T07:04:30.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 --2- 172.21.15.12:0/359085301 >> v2:172.21.15.53:3300/0 conn(0x7fbdb413e5b0 0x7fbdb413ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 --2- 172.21.15.12:0/359085301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb41487b0 0x7fbdb4073c60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- 172.21.15.12:0/359085301 >> 172.21.15.12:0/359085301 conn(0x7fbdb4067d10 msgr2=0x7fbdb4068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:30.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- 172.21.15.12:0/359085301 shutdown_connections 2024-04-27T07:04:30.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- 172.21.15.12:0/359085301 wait complete. 2024-04-27T07:04:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 Processor -- start 2024-04-27T07:04:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- start start 2024-04-27T07:04:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb40b9ae0 0x7fbdb40b8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbdb413e5b0 0x7fbdb41e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdb41487b0 0x7fbdb40ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb40b5110 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb409db90 con 0x7fbdb41487b0 2024-04-27T07:04:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb4151cb0 con 0x7fbdb413e5b0 2024-04-27T07:04:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb8a32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb40b9ae0 0x7fbdb40b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb8a32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb40b9ae0 0x7fbdb40b8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55744/0 (socket says 172.21.15.12:55744) 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb8a32640 1 -- 172.21.15.12:0/480017696 learned_addr learned my addr 172.21.15.12:0/480017696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb8a32640 1 -- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbdb413e5b0 msgr2=0x7fbdb41e3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb3fff640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbdb413e5b0 0x7fbdb41e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb8a32640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbdb413e5b0 0x7fbdb41e3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb9233640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdb41487b0 0x7fbdb40ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb8a32640 1 -- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdb41487b0 msgr2=0x7fbdb40ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb8a32640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdb41487b0 0x7fbdb40ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb8a32640 1 -- 172.21.15.12:0/480017696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb4141a40 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb8a32640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb40b9ae0 0x7fbdb40b8590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbda400c4b0 tx=0x7fbda400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb17fa640 1 -- 172.21.15.12:0/480017696 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4018070 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb17fa640 1 -- 172.21.15.12:0/480017696 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda4007500 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- 172.21.15.12:0/480017696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb413f290 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdb17fa640 1 -- 172.21.15.12:0/480017696 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4004a30 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- 172.21.15.12:0/480017696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb4141760 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.921+0000 7fbdbac96640 1 -- 172.21.15.12:0/480017696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb4068c20 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.925+0000 7fbdb17fa640 1 -- 172.21.15.12:0/480017696 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbda401b070 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.925+0000 7fbdb17fa640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbda0068380 0x7fbda006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:30.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.925+0000 7fbdb17fa640 1 -- 172.21.15.12:0/480017696 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbda4085c30 con 0x7fbdb40b9ae0 2024-04-27T07:04:30.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.925+0000 7fbdb3fff640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbda0068380 0x7fbda006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:30.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.925+0000 7fbdb3fff640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbda0068380 0x7fbda006a840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbda80ad000 tx=0x7fbda8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:30.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:30.929+0000 7fbdb17fa640 1 -- 172.21.15.12:0/480017696 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbda4056280 con 0x7fbdb40b9ae0 2024-04-27T07:04:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.093+0000 7fbdbac96640 1 -- 172.21.15.12:0/480017696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdb4142320 con 0x7fbdb40b9ae0 2024-04-27T07:04:31.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.093+0000 7fbdb17fa640 1 -- 172.21.15.12:0/480017696 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fbda4055b80 con 0x7fbdb40b9ae0 2024-04-27T07:04:31.099 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:31.100 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:31.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 -- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbda0068380 msgr2=0x7fbda006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbda0068380 0x7fbda006a840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbda80ad000 tx=0x7fbda8008040 comp rx=0 tx=0).stop 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 -- 172.21.15.12:0/480017696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb40b9ae0 msgr2=0x7fbdb40b8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb40b9ae0 0x7fbdb40b8590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbda400c4b0 tx=0x7fbda400c970 comp rx=0 tx=0).stop 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 -- 172.21.15.12:0/480017696 shutdown_connections 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbda0068380 0x7fbda006a840 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbdb41487b0 0x7fbdb40ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbdb413e5b0 0x7fbdb41e3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 --2- 172.21.15.12:0/480017696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbdb40b9ae0 0x7fbdb40b8590 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 -- 172.21.15.12:0/480017696 >> 172.21.15.12:0/480017696 conn(0x7fbdb4067d10 msgr2=0x7fbdb409b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 -- 172.21.15.12:0/480017696 shutdown_connections 2024-04-27T07:04:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.097+0000 7fbdbac96640 1 -- 172.21.15.12:0/480017696 wait complete. 2024-04-27T07:04:31.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:12.742740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:13.912898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:31.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f66744f3640 1 Processor -- start 2024-04-27T07:04:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f66744f3640 1 -- start start 2024-04-27T07:04:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f66744f3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f666c06f9e0 0x7f666c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f66744f3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666c06bb80 con 0x7f666c06c7a0 2024-04-27T07:04:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f66744f3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666c06be60 con 0x7f666c06f600 2024-04-27T07:04:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f66744f3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f666c1400e0 con 0x7f666c06f9e0 2024-04-27T07:04:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f6671a8e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f666c06c7a0 0x7f666c149230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59754/0 (socket says 172.21.15.12:59754) 2024-04-27T07:04:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f6671a8e640 1 -- 172.21.15.12:0/3419402328 learned_addr learned my addr 172.21.15.12:0/3419402328 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f6672a90640 1 --2- 172.21.15.12:0/3419402328 >> v2:172.21.15.12:3300/0 conn(0x7f666c06f9e0 0x7f666c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f667128d640 1 -- 172.21.15.12:0/3419402328 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682463982 0 0) 0x7f666c06bb80 con 0x7f666c06c7a0 2024-04-27T07:04:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f667128d640 1 -- 172.21.15.12:0/3419402328 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6658003530 con 0x7f666c06c7a0 2024-04-27T07:04:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f6672a90640 1 -- 172.21.15.12:0/3419402328 >> v1:172.21.15.53:6789/0 conn(0x7f666c06f600 legacy=0x7f666c0b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f6672a90640 1 -- 172.21.15.12:0/3419402328 >> v1:172.21.15.12:6789/0 conn(0x7f666c06c7a0 legacy=0x7f666c149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.249+0000 7f6672a90640 1 -- 172.21.15.12:0/3419402328 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c149940 con 0x7f666c06f9e0 2024-04-27T07:04:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f6672a90640 1 --2- 172.21.15.12:0/3419402328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06f9e0 0x7f666c1515d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6664009d90 tx=0x7f666401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c4dfc8e16ff2c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667128d640 1 -- 172.21.15.12:0/3419402328 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66640b2070 con 0x7f666c06f9e0 2024-04-27T07:04:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667128d640 1 -- 172.21.15.12:0/3419402328 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66640a58b0 con 0x7f666c06f9e0 2024-04-27T07:04:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667128d640 1 -- 172.21.15.12:0/3419402328 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66640a5c80 con 0x7f666c06f9e0 2024-04-27T07:04:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 -- 172.21.15.12:0/3419402328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06f9e0 msgr2=0x7f666c1515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 --2- 172.21.15.12:0/3419402328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06f9e0 0x7f666c1515d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6664009d90 tx=0x7f666401c8e0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 -- 172.21.15.12:0/3419402328 shutdown_connections 2024-04-27T07:04:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 --2- 172.21.15.12:0/3419402328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06f9e0 0x7f666c1515d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 -- 172.21.15.12:0/3419402328 >> 172.21.15.12:0/3419402328 conn(0x7f666c09c4f0 msgr2=0x7f666c09c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 -- 172.21.15.12:0/3419402328 shutdown_connections 2024-04-27T07:04:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 -- 172.21.15.12:0/3419402328 wait complete. 2024-04-27T07:04:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 Processor -- start 2024-04-27T07:04:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 -- start start 2024-04-27T07:04:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06c7a0 0x7f666c143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f666c06f600 0x7f666c0b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f666c06f9e0 0x7f666c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f666c154950 con 0x7f666c06c7a0 2024-04-27T07:04:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f666c149b10 con 0x7f666c06f600 2024-04-27T07:04:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f66744f3640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f666c154ad0 con 0x7f666c06f9e0 2024-04-27T07:04:31.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667228f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06c7a0 0x7f666c143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667228f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06c7a0 0x7f666c143b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55756/0 (socket says 172.21.15.12:55756) 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667228f640 1 -- 172.21.15.12:0/4025174843 learned_addr learned my addr 172.21.15.12:0/4025174843 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f6671a8e640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f666c06f600 0x7f666c0b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f6672a90640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f666c06f9e0 0x7f666c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667228f640 1 -- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f666c06f9e0 msgr2=0x7f666c0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667228f640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f666c06f9e0 0x7f666c0b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667228f640 1 -- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f666c06f600 msgr2=0x7f666c0b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667228f640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f666c06f600 0x7f666c0b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667228f640 1 -- 172.21.15.12:0/4025174843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c146920 con 0x7f666c06c7a0 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f667228f640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06c7a0 0x7f666c143b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f665c00c420 tx=0x7f665c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.253+0000 7f6662ffd640 1 -- 172.21.15.12:0/4025174843 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0052d0 con 0x7f666c06c7a0 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.257+0000 7f6662ffd640 1 -- 172.21.15.12:0/4025174843 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665c005470 con 0x7f666c06c7a0 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.257+0000 7f66744f3640 1 -- 172.21.15.12:0/4025174843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c146360 con 0x7f666c06c7a0 2024-04-27T07:04:31.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.257+0000 7f6662ffd640 1 -- 172.21.15.12:0/4025174843 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c007260 con 0x7f666c06c7a0 2024-04-27T07:04:31.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.257+0000 7f66744f3640 1 -- 172.21.15.12:0/4025174843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c0b8790 con 0x7f666c06c7a0 2024-04-27T07:04:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.257+0000 7f66744f3640 1 -- 172.21.15.12:0/4025174843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666c069330 con 0x7f666c06c7a0 2024-04-27T07:04:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.257+0000 7f6662ffd640 1 -- 172.21.15.12:0/4025174843 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f665c007400 con 0x7f666c06c7a0 2024-04-27T07:04:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.257+0000 7f6662ffd640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6650068380 0x7f665006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.257+0000 7f6662ffd640 1 -- 172.21.15.12:0/4025174843 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f665c085e30 con 0x7f666c06c7a0 2024-04-27T07:04:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.257+0000 7f6671a8e640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6650068380 0x7f665006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.257+0000 7f6671a8e640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6650068380 0x7f665006a840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6668007fd0 tx=0x7f6668009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:31.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.261+0000 7f6662ffd640 1 -- 172.21.15.12:0/4025174843 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f665c056480 con 0x7f666c06c7a0 2024-04-27T07:04:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.425+0000 7f66744f3640 1 -- 172.21.15.12:0/4025174843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f666c0b6fb0 con 0x7f666c06c7a0 2024-04-27T07:04:31.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.425+0000 7f6662ffd640 1 -- 172.21.15.12:0/4025174843 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f665c055d80 con 0x7f666c06c7a0 2024-04-27T07:04:31.431 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 -- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6650068380 msgr2=0x7f665006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6650068380 0x7f665006a840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6668007fd0 tx=0x7f6668009040 comp rx=0 tx=0).stop 2024-04-27T07:04:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 -- 172.21.15.12:0/4025174843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06c7a0 msgr2=0x7f666c143b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06c7a0 0x7f666c143b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f665c00c420 tx=0x7f665c00c8e0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 -- 172.21.15.12:0/4025174843 shutdown_connections 2024-04-27T07:04:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6650068380 0x7f665006a840 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f666c06f9e0 0x7f666c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f666c06f600 0x7f666c0b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 --2- 172.21.15.12:0/4025174843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f666c06c7a0 0x7f666c143b20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 -- 172.21.15.12:0/4025174843 >> 172.21.15.12:0/4025174843 conn(0x7f666c09c4f0 msgr2=0x7f666c06e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 -- 172.21.15.12:0/4025174843 shutdown_connections 2024-04-27T07:04:31.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.429+0000 7f66744f3640 1 -- 172.21.15.12:0/4025174843 wait complete. 2024-04-27T07:04:31.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:04:31.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa4159da640 1 Processor -- start 2024-04-27T07:04:31.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa4159da640 1 -- start start 2024-04-27T07:04:31.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa4159da640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa4100b4ae0 0x7fa4101454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa4159da640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa41006e410 0x7fa41014f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa4159da640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4101459f0 con 0x7fa4100b3db0 2024-04-27T07:04:31.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa4159da640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fa410144a80 con 0x7fa4100b4ae0 2024-04-27T07:04:31.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa4159da640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa4100b4ec0 con 0x7fa41006e410 2024-04-27T07:04:31.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa40effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa4100b4ae0 0x7fa4101454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa407fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa4100b3db0 0x7fa41009da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59762/0 (socket says 172.21.15.12:59762) 2024-04-27T07:04:31.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa407fff640 1 -- 172.21.15.12:0/3221556444 learned_addr learned my addr 172.21.15.12:0/3221556444 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:31.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa40f7fe640 1 --2- 172.21.15.12:0/3221556444 >> v2:172.21.15.53:3300/0 conn(0x7fa41006e410 0x7fa41014f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa40effd640 1 -- 172.21.15.12:0/3221556444 >> v2:172.21.15.53:3300/0 conn(0x7fa41006e410 msgr2=0x7fa41014f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:31.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa40effd640 1 --2- 172.21.15.12:0/3221556444 >> v2:172.21.15.53:3300/0 conn(0x7fa41006e410 0x7fa41014f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa40effd640 1 -- 172.21.15.12:0/3221556444 >> v1:172.21.15.12:6789/0 conn(0x7fa4100b3db0 legacy=0x7fa41009da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa40effd640 1 -- 172.21.15.12:0/3221556444 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa410144cb0 con 0x7fa4100b4ae0 2024-04-27T07:04:31.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa40e7fc640 1 -- 172.21.15.12:0/3221556444 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1135753426 0 0) 0x7fa4101459f0 con 0x7fa4100b3db0 2024-04-27T07:04:31.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.577+0000 7fa40f7fe640 1 --2- 172.21.15.12:0/3221556444 >> v2:172.21.15.53:3300/0 conn(0x7fa41006e410 0x7fa41014f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:31.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40effd640 1 --2- 172.21.15.12:0/3221556444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b4ae0 0x7fa4101454b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa410149090 tx=0x7fa40001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7dd4023fdca53e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:31.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40e7fc640 1 -- 172.21.15.12:0/3221556444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4000b1070 con 0x7fa4100b4ae0 2024-04-27T07:04:31.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40e7fc640 1 -- 172.21.15.12:0/3221556444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4000ac5f0 con 0x7fa4100b4ae0 2024-04-27T07:04:31.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40e7fc640 1 -- 172.21.15.12:0/3221556444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4000ac9c0 con 0x7fa4100b4ae0 2024-04-27T07:04:31.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- 172.21.15.12:0/3221556444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b4ae0 msgr2=0x7fa4101454b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 --2- 172.21.15.12:0/3221556444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b4ae0 0x7fa4101454b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa410149090 tx=0x7fa40001c480 comp rx=0 tx=0).stop 2024-04-27T07:04:31.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- 172.21.15.12:0/3221556444 shutdown_connections 2024-04-27T07:04:31.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 --2- 172.21.15.12:0/3221556444 >> v2:172.21.15.53:3300/0 conn(0x7fa41006e410 0x7fa41014f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 --2- 172.21.15.12:0/3221556444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b4ae0 0x7fa4101454b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- 172.21.15.12:0/3221556444 >> 172.21.15.12:0/3221556444 conn(0x7fa4100b93b0 msgr2=0x7fa4100b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:31.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- 172.21.15.12:0/3221556444 shutdown_connections 2024-04-27T07:04:31.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- 172.21.15.12:0/3221556444 wait complete. 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 Processor -- start 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- start start 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa41006e410 0x7fa4101e1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b3db0 0x7fa410070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4100b4ae0 0x7fa410070a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa410152430 con 0x7fa4100b3db0 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101522b0 con 0x7fa4100b4ae0 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fa4101525b0 con 0x7fa41006e410 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa407fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b3db0 0x7fa410070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa407fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b3db0 0x7fa410070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55784/0 (socket says 172.21.15.12:55784) 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa407fff640 1 -- 172.21.15.12:0/3326169272 learned_addr learned my addr 172.21.15.12:0/3326169272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:31.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40f7fe640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4100b4ae0 0x7fa410070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40effd640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa41006e410 0x7fa4101e1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa407fff640 1 -- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa41006e410 msgr2=0x7fa4101e1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa407fff640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa41006e410 0x7fa4101e1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa407fff640 1 -- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4100b4ae0 msgr2=0x7fa410070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa407fff640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4100b4ae0 0x7fa410070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa407fff640 1 -- 172.21.15.12:0/3326169272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4101e2730 con 0x7fa4100b3db0 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40f7fe640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4100b4ae0 0x7fa410070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40effd640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa41006e410 0x7fa4101e1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa407fff640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b3db0 0x7fa410070540 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c4b0 tx=0x7fa3f800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40cff9640 1 -- 172.21.15.12:0/3326169272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8018070 con 0x7fa4100b3db0 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40cff9640 1 -- 172.21.15.12:0/3326169272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f80074e0 con 0x7fa4100b3db0 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- 172.21.15.12:0/3326169272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa410072d60 con 0x7fa4100b3db0 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa40cff9640 1 -- 172.21.15.12:0/3326169272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8004a30 con 0x7fa4100b3db0 2024-04-27T07:04:31.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.581+0000 7fa4159da640 1 -- 172.21.15.12:0/3326169272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa410147dd0 con 0x7fa4100b3db0 2024-04-27T07:04:31.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.585+0000 7fa4159da640 1 -- 172.21.15.12:0/3326169272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410068c20 con 0x7fa4100b3db0 2024-04-27T07:04:31.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.585+0000 7fa40cff9640 1 -- 172.21.15.12:0/3326169272 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa3f801b070 con 0x7fa4100b3db0 2024-04-27T07:04:31.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.585+0000 7fa40cff9640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa3f00683b0 0x7fa3f006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.585+0000 7fa40cff9640 1 -- 172.21.15.12:0/3326169272 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3f8085c50 con 0x7fa4100b3db0 2024-04-27T07:04:31.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.585+0000 7fa40effd640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa3f00683b0 0x7fa3f006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.585+0000 7fa40effd640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa3f00683b0 0x7fa3f006a870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4000ae000 tx=0x7fa4000a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:31.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.589+0000 7fa40cff9640 1 -- 172.21.15.12:0/3326169272 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa3f808b020 con 0x7fa4100b3db0 2024-04-27T07:04:31.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.757+0000 7fa4159da640 1 -- 172.21.15.12:0/3326169272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa4100aeb90 con 0x7fa4100b3db0 2024-04-27T07:04:31.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.757+0000 7fa40cff9640 1 -- 172.21.15.12:0/3326169272 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fa3f8056210 con 0x7fa4100b3db0 2024-04-27T07:04:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:04:31.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 -- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa3f00683b0 msgr2=0x7fa3f006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa3f00683b0 0x7fa3f006a870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4000ae000 tx=0x7fa4000a7000 comp rx=0 tx=0).stop 2024-04-27T07:04:31.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 -- 172.21.15.12:0/3326169272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b3db0 msgr2=0x7fa410070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b3db0 0x7fa410070540 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c4b0 tx=0x7fa3f800c970 comp rx=0 tx=0).stop 2024-04-27T07:04:31.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 -- 172.21.15.12:0/3326169272 shutdown_connections 2024-04-27T07:04:31.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa3f00683b0 0x7fa3f006a870 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4000ae000 tx=0x7fa4000a7000 comp rx=0 tx=0).stop 2024-04-27T07:04:31.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4100b4ae0 0x7fa410070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4100b3db0 0x7fa410070540 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 --2- 172.21.15.12:0/3326169272 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa41006e410 0x7fa4101e1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 -- 172.21.15.12:0/3326169272 >> 172.21.15.12:0/3326169272 conn(0x7fa4100b93b0 msgr2=0x7fa4100b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:31.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 -- 172.21.15.12:0/3326169272 shutdown_connections 2024-04-27T07:04:31.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.761+0000 7fa4159da640 1 -- 172.21.15.12:0/3326169272 wait complete. 2024-04-27T07:04:31.782 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:04:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d83fe0640 1 Processor -- start 2024-04-27T07:04:31.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d83fe0640 1 -- start start 2024-04-27T07:04:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d83fe0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8d7c14b7d0 0x7f8d7c1483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d83fe0640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8d7c148910 0x7f8d7c152660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d83fe0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d7c0b4a30 con 0x7f8d7c0b4650 2024-04-27T07:04:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d83fe0640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f8d7c148d80 con 0x7f8d7c148910 2024-04-27T07:04:31.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d83fe0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8d7c14bc30 con 0x7f8d7c14b7d0 2024-04-27T07:04:31.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d8257d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8d7c148910 0x7f8d7c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d8257d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8d7c148910 0x7f8d7c152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55792/0 (socket says 172.21.15.12:55792) 2024-04-27T07:04:31.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d8257d640 1 -- 172.21.15.12:0/3270275835 learned_addr learned my addr 172.21.15.12:0/3270275835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:31.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.909+0000 7f8d8157b640 1 --2- 172.21.15.12:0/3270275835 >> v2:172.21.15.53:3300/0 conn(0x7f8d7c14b7d0 0x7f8d7c1483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d8257d640 1 -- 172.21.15.12:0/3270275835 >> v2:172.21.15.53:3300/0 conn(0x7f8d7c14b7d0 msgr2=0x7f8d7c1483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:31.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d8257d640 1 --2- 172.21.15.12:0/3270275835 >> v2:172.21.15.53:3300/0 conn(0x7f8d7c14b7d0 0x7f8d7c1483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d8257d640 1 -- 172.21.15.12:0/3270275835 >> v1:172.21.15.53:6789/0 conn(0x7f8d7c0b4650 legacy=0x7f8d7c073a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d8257d640 1 -- 172.21.15.12:0/3270275835 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c14c1d0 con 0x7f8d7c148910 2024-04-27T07:04:31.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d8257d640 1 --2- 172.21.15.12:0/3270275835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 0x7f8d7c152660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c069d80 tx=0x7f8d7401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1345713b9b04831e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:31.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d80d7a640 1 -- 172.21.15.12:0/3270275835 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740a5710 con 0x7f8d7c148910 2024-04-27T07:04:31.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d80d7a640 1 -- 172.21.15.12:0/3270275835 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d740a58b0 con 0x7f8d7c148910 2024-04-27T07:04:31.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d80d7a640 1 -- 172.21.15.12:0/3270275835 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740a5c80 con 0x7f8d7c148910 2024-04-27T07:04:31.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3270275835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 msgr2=0x7f8d7c152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d83fe0640 1 --2- 172.21.15.12:0/3270275835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 0x7f8d7c152660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c069d80 tx=0x7f8d7401c960 comp rx=0 tx=0).stop 2024-04-27T07:04:31.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3270275835 shutdown_connections 2024-04-27T07:04:31.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d83fe0640 1 --2- 172.21.15.12:0/3270275835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 0x7f8d7c152660 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d83fe0640 1 --2- 172.21.15.12:0/3270275835 >> v2:172.21.15.53:3300/0 conn(0x7f8d7c14b7d0 0x7f8d7c1483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3270275835 >> 172.21.15.12:0/3270275835 conn(0x7f8d7c0b93b0 msgr2=0x7f8d7c0b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:31.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3270275835 shutdown_connections 2024-04-27T07:04:31.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.913+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3270275835 wait complete. 2024-04-27T07:04:31.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 Processor -- start 2024-04-27T07:04:31.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 -- start start 2024-04-27T07:04:31.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c0b4650 0x7f8d7c145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 0x7f8d7c143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d7c14b7d0 0x7f8d7c142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c0b41b0 con 0x7f8d7c148910 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c0b4030 con 0x7f8d7c0b4650 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c1553c0 con 0x7f8d7c14b7d0 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d8157b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 0x7f8d7c143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d8157b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 0x7f8d7c143c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55796/0 (socket says 172.21.15.12:55796) 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d8157b640 1 -- 172.21.15.12:0/3565140208 learned_addr learned my addr 172.21.15.12:0/3565140208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d81d7c640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c0b4650 0x7f8d7c145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d8257d640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d7c14b7d0 0x7f8d7c142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d8157b640 1 -- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d7c14b7d0 msgr2=0x7f8d7c142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d8157b640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d7c14b7d0 0x7f8d7c142c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d8157b640 1 -- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c0b4650 msgr2=0x7f8d7c145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d8157b640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c0b4650 0x7f8d7c145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:31.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d8157b640 1 -- 172.21.15.12:0/3565140208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c1446b0 con 0x7f8d7c148910 2024-04-27T07:04:31.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d81d7c640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c0b4650 0x7f8d7c145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:31.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d8157b640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 0x7f8d7c143c10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00e930 tx=0x7f8d6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:31.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d727fc640 1 -- 172.21.15.12:0/3565140208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c017070 con 0x7f8d7c148910 2024-04-27T07:04:31.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d727fc640 1 -- 172.21.15.12:0/3565140208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c00ca00 con 0x7f8d7c148910 2024-04-27T07:04:31.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d727fc640 1 -- 172.21.15.12:0/3565140208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00cdb0 con 0x7f8d7c148910 2024-04-27T07:04:31.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3565140208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c0730d0 con 0x7f8d7c148910 2024-04-27T07:04:31.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3565140208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c145650 con 0x7f8d7c148910 2024-04-27T07:04:31.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.917+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3565140208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d7c068c20 con 0x7f8d7c148910 2024-04-27T07:04:31.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.921+0000 7f8d727fc640 1 -- 172.21.15.12:0/3565140208 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8d6c01a070 con 0x7f8d7c148910 2024-04-27T07:04:31.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.921+0000 7f8d727fc640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d600683b0 0x7f8d6006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:31.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.921+0000 7f8d727fc640 1 -- 172.21.15.12:0/3565140208 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8d6c086940 con 0x7f8d7c148910 2024-04-27T07:04:31.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.921+0000 7f8d81d7c640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d600683b0 0x7f8d6006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:31.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.921+0000 7f8d81d7c640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d600683b0 0x7f8d6006a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8d78001670 tx=0x7f8d78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:31.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:31.925+0000 7f8d727fc640 1 -- 172.21.15.12:0/3565140208 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8d6c057050 con 0x7f8d7c148910 2024-04-27T07:04:32.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.085+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3565140208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8d7c06cc30 con 0x7f8d7c148910 2024-04-27T07:04:32.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.085+0000 7f8d727fc640 1 -- 172.21.15.12:0/3565140208 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8d6c056950 con 0x7f8d7c148910 2024-04-27T07:04:32.092 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:04:32.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d600683b0 msgr2=0x7f8d6006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d600683b0 0x7f8d6006a870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8d78001670 tx=0x7f8d78009040 comp rx=0 tx=0).stop 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3565140208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 msgr2=0x7f8d7c143c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 0x7f8d7c143c10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00e930 tx=0x7f8d6c00edf0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3565140208 shutdown_connections 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d600683b0 0x7f8d6006a870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d7c14b7d0 0x7f8d7c142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7c148910 0x7f8d7c143c10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 --2- 172.21.15.12:0/3565140208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d7c0b4650 0x7f8d7c145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3565140208 >> 172.21.15.12:0/3565140208 conn(0x7f8d7c0b93b0 msgr2=0x7f8d7c073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3565140208 shutdown_connections 2024-04-27T07:04:32.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.089+0000 7f8d83fe0640 1 -- 172.21.15.12:0/3565140208 wait complete. 2024-04-27T07:04:32.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:04:32.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.217+0000 7fe54f1ed640 1 Processor -- start 2024-04-27T07:04:32.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.217+0000 7fe54f1ed640 1 -- start start 2024-04-27T07:04:32.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe54f1ed640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe5480b9a90 0x7fe5480b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe54f1ed640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5480b9e70 con 0x7fe5480b3660 2024-04-27T07:04:32.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe54f1ed640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe54806e890 con 0x7fe54814e400 2024-04-27T07:04:32.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe54f1ed640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe54806daf0 con 0x7fe5480b9a90 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe547fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe5480b3660 0x7fe548145b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59772/0 (socket says 172.21.15.12:59772) 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe547fff640 1 -- 172.21.15.12:0/1943322758 learned_addr learned my addr 172.21.15.12:0/1943322758 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe54cf89640 1 --2- 172.21.15.12:0/1943322758 >> v2:172.21.15.53:3300/0 conn(0x7fe5480b9a90 0x7fe5480b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2787630004 0 0) 0x7fe5480b9e70 con 0x7fe5480b3660 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe534003530 con 0x7fe5480b3660 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 488807736 0 0) 0x7fe534003530 con 0x7fe5480b3660 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 >> v2:172.21.15.53:3300/0 conn(0x7fe5480b9a90 msgr2=0x7fe5480b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 --2- 172.21.15.12:0/1943322758 >> v2:172.21.15.53:3300/0 conn(0x7fe5480b9a90 0x7fe5480b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 >> v1:172.21.15.53:6789/0 conn(0x7fe54814e400 legacy=0x7fe5481507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54806e090 con 0x7fe5480b3660 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fe538001680 con 0x7fe5480b3660 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 >> v1:172.21.15.12:6789/0 conn(0x7fe5480b3660 legacy=0x7fe548145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5340039c0 0x7fe534003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe534004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe5340052f0 0x7fe53400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe54806daf0 con 0x7fe534004810 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe53400c1c0 con 0x7fe5340039c0 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe53400c3c0 con 0x7fe5340052f0 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe547fff640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe534004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe54cf89640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5340039c0 0x7fe534003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe54d78a640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe5340052f0 0x7fe53400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe547fff640 1 -- 172.21.15.12:0/1943322758 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe5340052f0 msgr2=0x7fe53400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe547fff640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe5340052f0 0x7fe53400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe547fff640 1 -- 172.21.15.12:0/1943322758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5340039c0 msgr2=0x7fe534003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe547fff640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5340039c0 0x7fe534003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe547fff640 1 -- 172.21.15.12:0/1943322758 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe534003530 con 0x7fe534004810 2024-04-27T07:04:32.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe54d78a640 1 -- 172.21.15.12:0/1943322758 reap_dead start 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.221+0000 7fe547fff640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe534004c90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c210 tx=0x7fe53801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd660fcb399926ea server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538007d20 con 0x7fe534004810 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5380a9070 con 0x7fe534004810 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe5477fe640 1 -- 172.21.15.12:0/1943322758 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538005da0 con 0x7fe534004810 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- 172.21.15.12:0/1943322758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 msgr2=0x7fe534004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe534004c90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c210 tx=0x7fe53801cc40 comp rx=0 tx=0).stop 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- 172.21.15.12:0/1943322758 shutdown_connections 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 --2- 172.21.15.12:0/1943322758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe534004c90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- 172.21.15.12:0/1943322758 >> 172.21.15.12:0/1943322758 conn(0x7fe548067d10 msgr2=0x7fe548068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- 172.21.15.12:0/1943322758 shutdown_connections 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- 172.21.15.12:0/1943322758 wait complete. 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 Processor -- start 2024-04-27T07:04:32.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- start start 2024-04-27T07:04:32.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe5480afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5481e13a0 0x7fe54809caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe54813d630 0x7fe54813dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe548145b70 con 0x7fe534004810 2024-04-27T07:04:32.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe548074090 con 0x7fe5481e13a0 2024-04-27T07:04:32.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe548145cf0 con 0x7fe54813d630 2024-04-27T07:04:32.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54cf89640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe5480afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54cf89640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe5480afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55812/0 (socket says 172.21.15.12:55812) 2024-04-27T07:04:32.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54cf89640 1 -- 172.21.15.12:0/903477773 learned_addr learned my addr 172.21.15.12:0/903477773 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:32.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54d78a640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe54813d630 0x7fe54813dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54cf89640 1 -- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe54813d630 msgr2=0x7fe54813dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54cf89640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe54813d630 0x7fe54813dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54cf89640 1 -- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5481e13a0 msgr2=0x7fe54809caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe547fff640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5481e13a0 0x7fe54809caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54cf89640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5481e13a0 0x7fe54809caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54cf89640 1 -- 172.21.15.12:0/903477773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe548149c40 con 0x7fe534004810 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54cf89640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe5480afa40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe530011690 tx=0x7fe530011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe5457fa640 1 -- 172.21.15.12:0/903477773 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53000f9c0 con 0x7fe534004810 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe5457fa640 1 -- 172.21.15.12:0/903477773 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53000fb60 con 0x7fe534004810 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- 172.21.15.12:0/903477773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54809d030 con 0x7fe534004810 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe5457fa640 1 -- 172.21.15.12:0/903477773 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530015460 con 0x7fe534004810 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.225+0000 7fe54f1ed640 1 -- 172.21.15.12:0/903477773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5481495f0 con 0x7fe534004810 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.229+0000 7fe54f1ed640 1 -- 172.21.15.12:0/903477773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe548068c20 con 0x7fe534004810 2024-04-27T07:04:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.229+0000 7fe5457fa640 1 -- 172.21.15.12:0/903477773 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe530015600 con 0x7fe534004810 2024-04-27T07:04:32.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.229+0000 7fe5457fa640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe5280683b0 0x7fe52806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.229+0000 7fe5457fa640 1 -- 172.21.15.12:0/903477773 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe530087640 con 0x7fe534004810 2024-04-27T07:04:32.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.229+0000 7fe547fff640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe5280683b0 0x7fe52806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.229+0000 7fe547fff640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe5280683b0 0x7fe52806a870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5380ab000 tx=0x7fe5380a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:32.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.233+0000 7fe5457fa640 1 -- 172.21.15.12:0/903477773 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe530019030 con 0x7fe534004810 2024-04-27T07:04:32.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.425+0000 7fe54f1ed640 1 -- 172.21.15.12:0/903477773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe54806d7f0 con 0x7fe534004810 2024-04-27T07:04:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.425+0000 7fe5457fa640 1 -- 172.21.15.12:0/903477773 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3527 (secure 0 0 0) 0x7fe530057d70 con 0x7fe534004810 2024-04-27T07:04:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:32.431 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1908224930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1908224930},{"type":"v1","addr":"172.21.15.53:6835","nonce":1908224930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:04:14.961460+0000","modified":"2024-04-27T07:04:17.982838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4242":{"gid":4242,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4173401303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4173401303},{"type":"v1","addr":"172.21.15.12:6833","nonce":4173401303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4242,"qdb_cluster":[4242]},"id":1}]} 2024-04-27T07:04:32.431 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-04-27T07:04:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 -- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe5280683b0 msgr2=0x7fe52806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe5280683b0 0x7fe52806a870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5380ab000 tx=0x7fe5380a7070 comp rx=0 tx=0).stop 2024-04-27T07:04:32.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 -- 172.21.15.12:0/903477773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 msgr2=0x7fe5480afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe5480afa40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe530011690 tx=0x7fe530011b50 comp rx=0 tx=0).stop 2024-04-27T07:04:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 -- 172.21.15.12:0/903477773 shutdown_connections 2024-04-27T07:04:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe5280683b0 0x7fe52806a870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe54813d630 0x7fe54813dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe5481e13a0 0x7fe54809caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 --2- 172.21.15.12:0/903477773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe534004810 0x7fe5480afa40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 -- 172.21.15.12:0/903477773 >> 172.21.15.12:0/903477773 conn(0x7fe548067d10 msgr2=0x7fe54813f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 -- 172.21.15.12:0/903477773 shutdown_connections 2024-04-27T07:04:32.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.429+0000 7fe54f1ed640 1 -- 172.21.15.12:0/903477773 wait complete. 2024-04-27T07:04:32.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f5545f39640 1 Processor -- start 2024-04-27T07:04:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f5545f39640 1 -- start start 2024-04-27T07:04:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f5545f39640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 0x7f55400b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f5545f39640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5540148810 0x7f5540152640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f5545f39640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55400b7560 con 0x7f55400b64c0 2024-04-27T07:04:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f5545f39640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5540148ca0 con 0x7f5540148810 2024-04-27T07:04:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f5545f39640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f55400b77d0 con 0x7f55400b5ae0 2024-04-27T07:04:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f553ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5540148810 0x7f5540152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f553effd640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 0x7f55400b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f553effd640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 0x7f55400b7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:57012/0 (socket says 172.21.15.12:57012) 2024-04-27T07:04:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f553ffff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5540148810 0x7f5540152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:39344/0 (socket says 172.21.15.12:39344) 2024-04-27T07:04:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f553f7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f55400b64c0 0x7f5540141ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:35588/0 (socket says 172.21.15.12:35588) 2024-04-27T07:04:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f553effd640 1 -- 172.21.15.12:0/604725791 learned_addr learned my addr 172.21.15.12:0/604725791 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f553effd640 1 -- 172.21.15.12:0/604725791 >> v2:172.21.15.53:3300/0 conn(0x7f5540148810 msgr2=0x7f5540152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f553effd640 1 --2- 172.21.15.12:0/604725791 >> v2:172.21.15.53:3300/0 conn(0x7f5540148810 0x7f5540152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f553effd640 1 -- 172.21.15.12:0/604725791 >> v1:172.21.15.53:6789/0 conn(0x7f55400b64c0 legacy=0x7f5540141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.565+0000 7f553effd640 1 -- 172.21.15.12:0/604725791 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55400b68a0 con 0x7f55400b5ae0 2024-04-27T07:04:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f553effd640 1 --2- 172.21.15.12:0/604725791 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 0x7f55400b7f60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5530009870 tx=0x7f553001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23be7499cab8e8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f553e7fc640 1 -- 172.21.15.12:0/604725791 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300ab610 con 0x7f55400b5ae0 2024-04-27T07:04:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f553e7fc640 1 -- 172.21.15.12:0/604725791 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55300b0070 con 0x7f55400b5ae0 2024-04-27T07:04:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f553e7fc640 1 -- 172.21.15.12:0/604725791 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300ab9d0 con 0x7f55400b5ae0 2024-04-27T07:04:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f5545f39640 1 -- 172.21.15.12:0/604725791 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 msgr2=0x7f55400b7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f5545f39640 1 --2- 172.21.15.12:0/604725791 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 0x7f55400b7f60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5530009870 tx=0x7f553001c410 comp rx=0 tx=0).stop 2024-04-27T07:04:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f5545f39640 1 -- 172.21.15.12:0/604725791 shutdown_connections 2024-04-27T07:04:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f5545f39640 1 --2- 172.21.15.12:0/604725791 >> v2:172.21.15.53:3300/0 conn(0x7f5540148810 0x7f5540152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f5545f39640 1 --2- 172.21.15.12:0/604725791 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 0x7f55400b7f60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f5545f39640 1 -- 172.21.15.12:0/604725791 >> 172.21.15.12:0/604725791 conn(0x7f5540073c00 msgr2=0x7f55400bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f5545f39640 1 -- 172.21.15.12:0/604725791 shutdown_connections 2024-04-27T07:04:32.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f5545f39640 1 -- 172.21.15.12:0/604725791 wait complete. 2024-04-27T07:04:32.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f5545f39640 1 Processor -- start 2024-04-27T07:04:32.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.569+0000 7f5545f39640 1 -- start start 2024-04-27T07:04:32.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5545f39640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 0x7f55401e4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5545f39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55400b64c0 0x7f554013ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5545f39640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540148810 0x7f55401e0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5545f39640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f55400b51c0 con 0x7f55400b64c0 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5545f39640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f55400b5040 con 0x7f5540148810 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5545f39640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f55401421e0 con 0x7f55400b5ae0 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55400b64c0 0x7f554013ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55400b64c0 0x7f554013ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55814/0 (socket says 172.21.15.12:55814) 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553effd640 1 -- 172.21.15.12:0/1319777728 learned_addr learned my addr 172.21.15.12:0/1319777728 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553ffff640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540148810 0x7f55401e0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553f7fe640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 0x7f55401e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553effd640 1 -- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 msgr2=0x7f55401e4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553effd640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 0x7f55401e4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553effd640 1 -- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540148810 msgr2=0x7f55401e0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553effd640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540148810 0x7f55401e0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553effd640 1 -- 172.21.15.12:0/1319777728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55401e3890 con 0x7f55400b64c0 2024-04-27T07:04:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553ffff640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540148810 0x7f55401e0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553effd640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55400b64c0 0x7f554013ff50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55300ad040 tx=0x7f553001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5537fff640 1 -- 172.21.15.12:0/1319777728 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300be5f0 con 0x7f55400b64c0 2024-04-27T07:04:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5537fff640 1 -- 172.21.15.12:0/1319777728 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55300be7b0 con 0x7f55400b64c0 2024-04-27T07:04:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5545f39640 1 -- 172.21.15.12:0/1319777728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554014a020 con 0x7f55400b64c0 2024-04-27T07:04:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5537fff640 1 -- 172.21.15.12:0/1319777728 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300beba0 con 0x7f55400b64c0 2024-04-27T07:04:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5545f39640 1 -- 172.21.15.12:0/1319777728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55401e3e10 con 0x7f55400b64c0 2024-04-27T07:04:32.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5545f39640 1 -- 172.21.15.12:0/1319777728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5540068c20 con 0x7f55400b64c0 2024-04-27T07:04:32.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5537fff640 1 -- 172.21.15.12:0/1319777728 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f55300b43f0 con 0x7f55400b64c0 2024-04-27T07:04:32.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5537fff640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5520068380 0x7f552006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f5537fff640 1 -- 172.21.15.12:0/1319777728 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5530121810 con 0x7f55400b64c0 2024-04-27T07:04:32.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.573+0000 7f553f7fe640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5520068380 0x7f552006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.577+0000 7f553f7fe640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5520068380 0x7f552006a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5528007fd0 tx=0x7f5528005b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:32.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.577+0000 7f5537fff640 1 -- 172.21.15.12:0/1319777728 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55300a8050 con 0x7f55400b64c0 2024-04-27T07:04:32.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.745+0000 7f5545f39640 1 -- 172.21.15.12:0/1319777728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f554013a150 con 0x7f55400b64c0 2024-04-27T07:04:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.745+0000 7f5537fff640 1 -- 172.21.15.12:0/1319777728 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f5530016150 con 0x7f55400b64c0 2024-04-27T07:04:32.751 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:32.751 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:32.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 -- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5520068380 msgr2=0x7f552006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5520068380 0x7f552006a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5528007fd0 tx=0x7f5528005b80 comp rx=0 tx=0).stop 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 -- 172.21.15.12:0/1319777728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55400b64c0 msgr2=0x7f554013ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55400b64c0 0x7f554013ff50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55300ad040 tx=0x7f553001ac50 comp rx=0 tx=0).stop 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 -- 172.21.15.12:0/1319777728 shutdown_connections 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5520068380 0x7f552006a840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5540148810 0x7f55401e0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55400b64c0 0x7f554013ff50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 --2- 172.21.15.12:0/1319777728 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55400b5ae0 0x7f55401e4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 -- 172.21.15.12:0/1319777728 >> 172.21.15.12:0/1319777728 conn(0x7f5540073c00 msgr2=0x7f5540140b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 -- 172.21.15.12:0/1319777728 shutdown_connections 2024-04-27T07:04:32.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.749+0000 7f5545f39640 1 -- 172.21.15.12:0/1319777728 wait complete. 2024-04-27T07:04:32.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:04:32.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eb91c3640 1 Processor -- start 2024-04-27T07:04:32.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eb91c3640 1 -- start start 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eb91c3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7eb40b54a0 0x7f7eb4148650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eb91c3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb4147b90 con 0x7f7eb40b6560 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eb91c3640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb40b7760 con 0x7f7eb4142020 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eb91c3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb41481c0 con 0x7f7eb40b54a0 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eabfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7eb40b54a0 0x7f7eb4148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eabfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7eb40b54a0 0x7f7eb4148650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55826/0 (socket says 172.21.15.12:55826) 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eabfff640 1 -- 172.21.15.12:0/3229385614 learned_addr learned my addr 172.21.15.12:0/3229385614 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eb2d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7eb40b6560 0x7f7eb4141910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59788/0 (socket says 172.21.15.12:59788) 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eabfff640 1 -- 172.21.15.12:0/3229385614 >> v1:172.21.15.53:6789/0 conn(0x7f7eb4142020 legacy=0x7f7eb4152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eabfff640 1 -- 172.21.15.12:0/3229385614 >> v1:172.21.15.12:6789/0 conn(0x7f7eb40b6560 legacy=0x7f7eb4141910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eabfff640 1 -- 172.21.15.12:0/3229385614 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb4147e50 con 0x7f7eb40b54a0 2024-04-27T07:04:32.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eabfff640 1 --2- 172.21.15.12:0/3229385614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb40b54a0 0x7f7eb4148650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009870 tx=0x7f7e9c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b101a391014237b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eb259c640 1 -- 172.21.15.12:0/3229385614 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0b0070 con 0x7f7eb40b54a0 2024-04-27T07:04:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eb259c640 1 -- 172.21.15.12:0/3229385614 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c0ab600 con 0x7f7eb40b54a0 2024-04-27T07:04:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.877+0000 7f7eb259c640 1 -- 172.21.15.12:0/3229385614 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0ab9d0 con 0x7f7eb40b54a0 2024-04-27T07:04:32.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3229385614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb40b54a0 msgr2=0x7f7eb4148650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 --2- 172.21.15.12:0/3229385614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb40b54a0 0x7f7eb4148650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009870 tx=0x7f7e9c01c490 comp rx=0 tx=0).stop 2024-04-27T07:04:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3229385614 shutdown_connections 2024-04-27T07:04:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 --2- 172.21.15.12:0/3229385614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb40b54a0 0x7f7eb4148650 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3229385614 >> 172.21.15.12:0/3229385614 conn(0x7f7eb40bd5c0 msgr2=0x7f7eb40bd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3229385614 shutdown_connections 2024-04-27T07:04:32.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3229385614 wait complete. 2024-04-27T07:04:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 Processor -- start 2024-04-27T07:04:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- start start 2024-04-27T07:04:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7eb40b6560 0x7f7eb4136fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb4142020 0x7f7eb413d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb413d5b0 0x7f7eb4131810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb41567e0 con 0x7f7eb413d5b0 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb4156660 con 0x7f7eb4142020 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb4156960 con 0x7f7eb40b6560 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb413d5b0 0x7f7eb4131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb413d5b0 0x7f7eb4131810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55840/0 (socket says 172.21.15.12:55840) 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb359e640 1 -- 172.21.15.12:0/3826192008 learned_addr learned my addr 172.21.15.12:0/3826192008 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb2d9d640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7eb40b6560 0x7f7eb4136fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eabfff640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb4142020 0x7f7eb413d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb359e640 1 -- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7eb40b6560 msgr2=0x7f7eb4136fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb359e640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7eb40b6560 0x7f7eb4136fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb359e640 1 -- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb4142020 msgr2=0x7f7eb413d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb359e640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb4142020 0x7f7eb413d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb359e640 1 -- 172.21.15.12:0/3826192008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb413e580 con 0x7f7eb413d5b0 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb2d9d640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7eb40b6560 0x7f7eb4136fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eabfff640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb4142020 0x7f7eb413d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb359e640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb413d5b0 0x7f7eb4131810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00ba10 tx=0x7f7eac00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:32.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eab7fe640 1 -- 172.21.15.12:0/3826192008 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac019070 con 0x7f7eb413d5b0 2024-04-27T07:04:32.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eab7fe640 1 -- 172.21.15.12:0/3826192008 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eac00cb40 con 0x7f7eb413d5b0 2024-04-27T07:04:32.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3826192008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb413e860 con 0x7f7eb413d5b0 2024-04-27T07:04:32.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eab7fe640 1 -- 172.21.15.12:0/3826192008 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac014440 con 0x7f7eb413d5b0 2024-04-27T07:04:32.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3826192008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb413f100 con 0x7f7eb413d5b0 2024-04-27T07:04:32.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.881+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3826192008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb4068c20 con 0x7f7eb413d5b0 2024-04-27T07:04:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.885+0000 7f7eab7fe640 1 -- 172.21.15.12:0/3826192008 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7eac01c070 con 0x7f7eb413d5b0 2024-04-27T07:04:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.885+0000 7f7eab7fe640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e94068380 0x7f7e9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.885+0000 7f7eab7fe640 1 -- 172.21.15.12:0/3826192008 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7eac0865d0 con 0x7f7eb413d5b0 2024-04-27T07:04:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.885+0000 7f7eb2d9d640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e94068380 0x7f7e9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.885+0000 7f7eb2d9d640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e94068380 0x7f7e9406a840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4145ad0 tx=0x7f7ea4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:32.889+0000 7f7eab7fe640 1 -- 172.21.15.12:0/3826192008 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7eac056c20 con 0x7f7eb413d5b0 2024-04-27T07:04:33.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.077+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3826192008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7eb40b7160 con 0x7f7eb413d5b0 2024-04-27T07:04:33.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eab7fe640 1 -- 172.21.15.12:0/3826192008 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3527 (secure 0 0 0) 0x7f7eac056520 con 0x7f7eb413d5b0 2024-04-27T07:04:33.085 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:33.085 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1908224930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1908224930},{"type":"v1","addr":"172.21.15.53:6835","nonce":1908224930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:04:14.961460+0000","modified":"2024-04-27T07:04:17.982838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4242":{"gid":4242,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4173401303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4173401303},{"type":"v1","addr":"172.21.15.12:6833","nonce":4173401303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4242,"qdb_cluster":[4242]},"id":1}]} 2024-04-27T07:04:33.085 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-04-27T07:04:33.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e94068380 msgr2=0x7f7e9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e94068380 0x7f7e9406a840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4145ad0 tx=0x7f7ea4009040 comp rx=0 tx=0).stop 2024-04-27T07:04:33.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3826192008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb413d5b0 msgr2=0x7f7eb4131810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb413d5b0 0x7f7eb4131810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00ba10 tx=0x7f7eac00bed0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3826192008 shutdown_connections 2024-04-27T07:04:33.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e94068380 0x7f7e9406a840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb413d5b0 0x7f7eb4131810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7eb4142020 0x7f7eb413d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 --2- 172.21.15.12:0/3826192008 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7eb40b6560 0x7f7eb4136fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3826192008 >> 172.21.15.12:0/3826192008 conn(0x7f7eb40bd5c0 msgr2=0x7f7eb40b83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:33.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3826192008 shutdown_connections 2024-04-27T07:04:33.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.081+0000 7f7eb91c3640 1 -- 172.21.15.12:0/3826192008 wait complete. 2024-04-27T07:04:33.304 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-27T07:04:14.961460+0000', 'modified': '2024-04-27T07:04:17.982838+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4242': {'gid': 4242, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.12:6833/4173401303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6832', 'nonce': 4173401303}, {'type': 'v1', 'addr': '172.21.15.12:6833', 'nonce': 4173401303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4242, 'qdb_cluster': [4242]} 2024-04-27T07:04:33.304 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-27T07:04:33.304 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-27T07:04:33.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24ade97640 1 Processor -- start 2024-04-27T07:04:33.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24ade97640 1 -- start start 2024-04-27T07:04:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24ade97640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8067b70 0x7f24a8143120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24ade97640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a8068870 con 0x7f24a80b31f0 2024-04-27T07:04:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24ade97640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a8067f50 con 0x7f24a8143660 2024-04-27T07:04:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24ade97640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f24a80b2650 con 0x7f24a8067b70 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a7fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f24a80b31f0 0x7f24a81504e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59800/0 (socket says 172.21.15.12:59800) 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a7fff640 1 -- 172.21.15.12:0/3348626374 learned_addr learned my addr 172.21.15.12:0/3348626374 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a77fe640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8067b70 0x7f24a8143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3307589024 0 0) 0x7f24a8068870 con 0x7f24a80b31f0 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2490003530 con 0x7f24a80b31f0 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 495856544 0 0) 0x7f2490003530 con 0x7f24a80b31f0 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8067b70 msgr2=0x7f24a8143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8067b70 0x7f24a8143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 >> v1:172.21.15.53:6789/0 conn(0x7f24a8143660 legacy=0x7f24a80b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a8072f70 con 0x7f24a80b31f0 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f24a0001a00 con 0x7f24a80b31f0 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 >> v1:172.21.15.12:6789/0 conn(0x7f24a80b31f0 legacy=0x7f24a81504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8067b70 0x7f24900039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490003f70 0x7f2490004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2490004d50 0x7f24900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24a80b2650 con 0x7f2490004d50 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f249000bb60 con 0x7f2490003f70 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f249000bd60 con 0x7f24a8067b70 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a7fff640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2490004d50 0x7f24900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a77fe640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8067b70 0x7f24900039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490003f70 0x7f2490004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a7fff640 1 -- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8067b70 msgr2=0x7f24900039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a7fff640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8067b70 0x7f24900039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a7fff640 1 -- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490003f70 msgr2=0x7f2490004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.429+0000 7f24a7fff640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490003f70 0x7f2490004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24a7fff640 1 -- 172.21.15.12:0/3348626374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2490003530 con 0x7f2490004d50 2024-04-27T07:04:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490003f70 0x7f2490004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24a77fe640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8067b70 0x7f24900039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24a7fff640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2490004d50 0x7f24900051d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24a000a840 tx=0x7f24a001fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a1a86e99edf287 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a000a460 con 0x7f2490004d50 2024-04-27T07:04:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a00aa710 con 0x7f2490004d50 2024-04-27T07:04:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24a67fc640 1 -- 172.21.15.12:0/3348626374 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a00aab00 con 0x7f2490004d50 2024-04-27T07:04:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24ade97640 1 -- 172.21.15.12:0/3348626374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2490004d50 msgr2=0x7f24900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24ade97640 1 --2- 172.21.15.12:0/3348626374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2490004d50 0x7f24900051d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24a000a840 tx=0x7f24a001fac0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24a7fff640 1 -- 172.21.15.12:0/3348626374 reap_dead start 2024-04-27T07:04:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24ade97640 1 -- 172.21.15.12:0/3348626374 shutdown_connections 2024-04-27T07:04:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24ade97640 1 -- 172.21.15.12:0/3348626374 >> 172.21.15.12:0/3348626374 conn(0x7f24a80b67b0 msgr2=0x7f24a80b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24ade97640 1 -- 172.21.15.12:0/3348626374 shutdown_connections 2024-04-27T07:04:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.433+0000 7f24ade97640 1 -- 172.21.15.12:0/3348626374 wait complete. 2024-04-27T07:04:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24ade97640 1 Processor -- start 2024-04-27T07:04:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24ade97640 1 -- start start 2024-04-27T07:04:33.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24ade97640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a80ae360 0x7f24a81e06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24ade97640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8072ab0 0x7f24a81e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24ade97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a8148f70 0x7f24a81493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24ade97640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24a80b6520 con 0x7f24a80ae360 2024-04-27T07:04:33.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24ade97640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24a80735a0 con 0x7f24a8148f70 2024-04-27T07:04:33.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24ade97640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f24a80b3a90 con 0x7f24a8072ab0 2024-04-27T07:04:33.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a80ae360 0x7f24a81e06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a80ae360 0x7f24a81e06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55852/0 (socket says 172.21.15.12:55852) 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a77fe640 1 -- 172.21.15.12:0/2446781596 learned_addr learned my addr 172.21.15.12:0/2446781596 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a7fff640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a8148f70 0x7f24a81493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8072ab0 0x7f24a81e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a77fe640 1 -- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8072ab0 msgr2=0x7f24a81e0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a77fe640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8072ab0 0x7f24a81e0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a77fe640 1 -- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a8148f70 msgr2=0x7f24a81493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a77fe640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a8148f70 0x7f24a81493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a77fe640 1 -- 172.21.15.12:0/2446781596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a80afda0 con 0x7f24a80ae360 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24a77fe640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a80ae360 0x7f24a81e06f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2498010900 tx=0x7f2498010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f248ffff640 1 -- 172.21.15.12:0/2446781596 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249800f040 con 0x7f24a80ae360 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f248ffff640 1 -- 172.21.15.12:0/2446781596 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249800ec20 con 0x7f24a80ae360 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24ade97640 1 -- 172.21.15.12:0/2446781596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a80aea90 con 0x7f24a80ae360 2024-04-27T07:04:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f248ffff640 1 -- 172.21.15.12:0/2446781596 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498004020 con 0x7f24a80ae360 2024-04-27T07:04:33.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f24ade97640 1 -- 172.21.15.12:0/2446781596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a80ad8f0 con 0x7f24a80ae360 2024-04-27T07:04:33.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f248ffff640 1 -- 172.21.15.12:0/2446781596 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f24980041c0 con 0x7f24a80ae360 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f248ffff640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f24940683b0 0x7f249406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.437+0000 7f248ffff640 1 -- 172.21.15.12:0/2446781596 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2498086d50 con 0x7f24a80ae360 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f24940683b0 0x7f249406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 Processor -- start 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- start start 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561984541f80 0x56198453d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f24940683b0 0x7f249406a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f24a80b7120 tx=0x7f249c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984542360 0x56198453d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56198453da80 0x56198454ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561984361be0 con 0x561984542360 2024-04-27T07:04:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5619842fbbf0 con 0x561984541f80 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x56198435dba0 con 0x56198453da80 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984542360 0x56198453d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984542360 0x56198453d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55858/0 (socket says 172.21.15.12:55858) 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 -- 172.21.15.12:0/1360866814 learned_addr learned my addr 172.21.15.12:0/1360866814 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a77fe640 1 --2- 172.21.15.12:0/1360866814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561984541f80 0x56198453d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a7fff640 1 --2- 172.21.15.12:0/1360866814 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56198453da80 0x56198454ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 -- 172.21.15.12:0/1360866814 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56198453da80 msgr2=0x56198454ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/1360866814 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56198453da80 0x56198454ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 -- 172.21.15.12:0/1360866814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561984541f80 msgr2=0x56198453d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/1360866814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561984541f80 0x56198453d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 -- 172.21.15.12:0/1360866814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561984409630 con 0x561984542360 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a7fff640 1 --2- 172.21.15.12:0/1360866814 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56198453da80 0x56198454ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/1360866814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984542360 0x56198453d540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f249c00e9c0 tx=0x7f249c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cddebab67c047ee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f248d7fa640 1 -- 172.21.15.12:0/1360866814 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c024070 con 0x561984542360 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f248d7fa640 1 -- 172.21.15.12:0/1360866814 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c01b040 con 0x561984542360 2024-04-27T07:04:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f248d7fa640 1 -- 172.21.15.12:0/1360866814 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c01a050 con 0x561984542360 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- 172.21.15.12:0/1360866814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984542360 msgr2=0x56198453d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 --2- 172.21.15.12:0/1360866814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984542360 0x56198453d540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f249c00e9c0 tx=0x7f249c011000 comp rx=0 tx=0).stop 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- 172.21.15.12:0/1360866814 shutdown_connections 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 --2- 172.21.15.12:0/1360866814 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x56198453da80 0x56198454ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 --2- 172.21.15.12:0/1360866814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984542360 0x56198453d540 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 --2- 172.21.15.12:0/1360866814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561984541f80 0x56198453d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- 172.21.15.12:0/1360866814 >> 172.21.15.12:0/1360866814 conn(0x56198450aa70 msgr2=0x561984541870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- 172.21.15.12:0/1360866814 shutdown_connections 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- 172.21.15.12:0/1360866814 wait complete. 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- start start 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 Processor -- start 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56198453da80 0x56198454fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561984541f80 0x561984550140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984515070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56198441dd10 con 0x56198453da80 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56198430b3a0 con 0x561984541f80 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x5619843251c0 con 0x561984542360 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.441+0000 7f24b1018000 10 client.0 _ll_register_callbacks cb 0x5619844fe730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56198453da80 0x56198454fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56198453da80 0x56198454fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55860/0 (socket says 172.21.15.12:55860) 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a77fe640 1 -- 172.21.15.12:0/1458080717 learned_addr learned my addr 172.21.15.12:0/1458080717 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561984541f80 0x561984550140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a7fff640 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984515070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a77fe640 1 -- 172.21.15.12:0/1458080717 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 msgr2=0x561984515070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a77fe640 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984515070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a77fe640 1 -- 172.21.15.12:0/1458080717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561984541f80 msgr2=0x561984550140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a77fe640 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561984541f80 0x561984550140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a77fe640 1 -- 172.21.15.12:0/1458080717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619843a70d0 con 0x56198453da80 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a7fff640 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984515070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a77fe640 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56198453da80 0x56198454fc00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f249805f2b0 tx=0x7f24980045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3582d1a16916f75f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24867fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24867fc640 1 -- 172.21.15.12:0/1458080717 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498021070 con 0x56198453da80 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24867fc640 1 -- 172.21.15.12:0/1458080717 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f249807edd0 con 0x56198453da80 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 -- 172.21.15.12:0/1458080717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f249c01b040 con 0x56198453da80 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24867fc640 1 -- 172.21.15.12:0/1458080717 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f249806f1d0 con 0x56198453da80 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 10 client.4668 fetch_fsmap learned FSMap version 8 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 -- 172.21.15.12:0/1458080717 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5619842e7e80 con 0x56198453da80 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24867fc640 1 -- 172.21.15.12:0/1458080717 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c01b040 con 0x56198453da80 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24867fc640 1 -- 172.21.15.12:0/1458080717 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 8) ==== 2284+0+0 (secure 0 0 0) 0x7f2498056050 con 0x56198453da80 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 10 client.4668 fetch_fsmap finished waiting for FSMap version 8 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 10 client.4668 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 20 client.4668 populate_metadata read hostname 'smithi012' 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.12:6832/4173401303,v1:172.21.15.12:6833/4173401303] conn(0x561984515e10 0x5619845161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 4 client.4668 mds_command: new command op to 4242 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 -- 172.21.15.12:0/1458080717 --> [v2:172.21.15.12:6832/4173401303,v1:172.21.15.12:6833/4173401303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619842fabc0 con 0x561984515e10 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.12:6832/4173401303,v1:172.21.15.12:6833/4173401303] conn(0x561984515e10 0x5619845161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24867fc640 10 client.4668 ms_handle_connect on v2:172.21.15.12:6832/4173401303 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.12:6832/4173401303,v1:172.21.15.12:6833/4173401303] conn(0x561984515e10 0x5619845161f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc7863355594809a server_cookie=eae9533c4d0d8670 in_seq=0 out_seq=0 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24867fc640 1 -- 172.21.15.12:0/1458080717 <== mds.0 v2:172.21.15.12:6832/4173401303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619842fabc0 con 0x561984515e10 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24867fc640 10 client.4668 handle_command_reply: tid=0 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 client.4668 shutdown 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 -- 172.21.15.12:0/1458080717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56198453da80 msgr2=0x56198454fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56198453da80 0x56198454fc00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f249805f2b0 tx=0x7f24980045f0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 -- 172.21.15.12:0/1458080717 shutdown_connections 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984515070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561984541f80 0x561984550140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.12:6832/4173401303,v1:172.21.15.12:6833/4173401303] conn(0x561984515e10 0x5619845161f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 --2- 172.21.15.12:0/1458080717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56198453da80 0x56198454fc00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24867fc640 0 client.4668 ms_handle_reset on v2:172.21.15.12:6832/4173401303 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 -- 172.21.15.12:0/1458080717 >> 172.21.15.12:0/1458080717 conn(0x56198450aa70 msgr2=0x561984541870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 -- 172.21.15.12:0/1458080717 shutdown_connections 2024-04-27T07:04:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 1 -- 172.21.15.12:0/1458080717 wait complete. 2024-04-27T07:04:33.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.445+0000 7f24b1018000 20 client.4668 trim_cache size 0 max 16384 2024-04-27T07:04:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 Processor -- start 2024-04-27T07:04:33.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- start start 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56198453da80 0x56198455fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x561984518910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984518e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81eaa90 con 0x561984541f80 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f24a81e9bf0 con 0x56198453da80 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x56198436cd00 con 0x561984542360 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x561984518910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x561984518910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55874/0 (socket says 172.21.15.12:55874) 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24a6ffd640 1 -- 172.21.15.12:0/2120162696 learned_addr learned my addr 172.21.15.12:0/2120162696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24a6ffd640 1 -- 172.21.15.12:0/2120162696 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 msgr2=0x561984518e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/2120162696 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984518e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24a6ffd640 1 -- 172.21.15.12:0/2120162696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56198453da80 msgr2=0x56198455fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/2120162696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56198453da80 0x56198455fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24a6ffd640 1 -- 172.21.15.12:0/2120162696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56198453e9a0 con 0x561984541f80 2024-04-27T07:04:33.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/2120162696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x561984518910 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f249c001ec0 tx=0x7f249c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c327ae21b79dbf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f248cff9640 1 -- 172.21.15.12:0/2120162696 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c024070 con 0x561984541f80 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f248cff9640 1 -- 172.21.15.12:0/2120162696 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c025040 con 0x561984541f80 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f248cff9640 1 -- 172.21.15.12:0/2120162696 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c01a040 con 0x561984541f80 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- 172.21.15.12:0/2120162696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 msgr2=0x561984518910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 --2- 172.21.15.12:0/2120162696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x561984518910 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f249c001ec0 tx=0x7f249c013000 comp rx=0 tx=0).stop 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- 172.21.15.12:0/2120162696 shutdown_connections 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 --2- 172.21.15.12:0/2120162696 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984518e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 --2- 172.21.15.12:0/2120162696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x561984518910 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 --2- 172.21.15.12:0/2120162696 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56198453da80 0x56198455fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- 172.21.15.12:0/2120162696 >> 172.21.15.12:0/2120162696 conn(0x56198450aa70 msgr2=0x561984518200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- 172.21.15.12:0/2120162696 shutdown_connections 2024-04-27T07:04:33.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- 172.21.15.12:0/2120162696 wait complete. 2024-04-27T07:04:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- start start 2024-04-27T07:04:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 Processor -- start 2024-04-27T07:04:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56198453da80 0x561984553170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x5619845536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984553bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561984366210 con 0x561984541f80 2024-04-27T07:04:33.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561984360ef0 con 0x56198453da80 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x56198436efe0 con 0x561984542360 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.465+0000 7f24b1018000 10 client.0 _ll_register_callbacks cb 0x7f24980096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x5619845536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x5619845536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55882/0 (socket says 172.21.15.12:55882) 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a6ffd640 1 -- 172.21.15.12:0/3089146154 learned_addr learned my addr 172.21.15.12:0/3089146154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a6ffd640 1 -- 172.21.15.12:0/3089146154 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 msgr2=0x561984553bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984553bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a6ffd640 1 -- 172.21.15.12:0/3089146154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56198453da80 msgr2=0x561984553170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56198453da80 0x561984553170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a6ffd640 1 -- 172.21.15.12:0/3089146154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561984329b30 con 0x561984541f80 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a6ffd640 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x5619845536b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f249c0052a0 tx=0x7f249c0011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d77e88c80a9d00d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24867fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24867fc640 1 -- 172.21.15.12:0/3089146154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c01a070 con 0x561984541f80 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24867fc640 1 -- 172.21.15.12:0/3089146154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f249c033d50 con 0x561984541f80 2024-04-27T07:04:33.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 -- 172.21.15.12:0/3089146154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f249c025040 con 0x561984541f80 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24867fc640 1 -- 172.21.15.12:0/3089146154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f249c044050 con 0x561984541f80 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 10 client.4674 fetch_fsmap learned FSMap version 8 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 -- 172.21.15.12:0/3089146154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5619843e2a10 con 0x561984541f80 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24867fc640 1 -- 172.21.15.12:0/3089146154 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c025040 con 0x561984541f80 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24867fc640 1 -- 172.21.15.12:0/3089146154 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 8) ==== 2284+0+0 (secure 0 0 0) 0x7f249c047020 con 0x561984541f80 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 10 client.4674 fetch_fsmap finished waiting for FSMap version 8 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 10 client.4674 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 20 client.4674 populate_metadata read hostname 'smithi012' 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.12:6832/4173401303,v1:172.21.15.12:6833/4173401303] conn(0x5619845be770 0x5619845c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 4 client.4674 mds_command: new command op to 4242 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 -- 172.21.15.12:0/3089146154 --> [v2:172.21.15.12:6832/4173401303,v1:172.21.15.12:6833/4173401303] -- command(tid 0: {"prefix": "status"}) -- 0x56198436f7f0 con 0x5619845be770 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a77fe640 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.12:6832/4173401303,v1:172.21.15.12:6833/4173401303] conn(0x5619845be770 0x5619845c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24a77fe640 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.12:6832/4173401303,v1:172.21.15.12:6833/4173401303] conn(0x5619845be770 0x5619845c0b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f455473b441dd676 server_cookie=690a29c1904fca6c in_seq=0 out_seq=0 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24867fc640 10 client.4674 ms_handle_connect on v2:172.21.15.12:6832/4173401303 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24867fc640 1 -- 172.21.15.12:0/3089146154 <== mds.0 v2:172.21.15.12:6832/4173401303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x56198436f7f0 con 0x5619845be770 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24867fc640 10 client.4674 handle_command_reply: tid=0 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 client.4674 shutdown 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 -- 172.21.15.12:0/3089146154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 msgr2=0x5619845536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x5619845536b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f249c0052a0 tx=0x7f249c0011a0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 -- 172.21.15.12:0/3089146154 shutdown_connections 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x561984542360 0x561984553bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.12:6832/4173401303,v1:172.21.15.12:6833/4173401303] conn(0x5619845be770 0x5619845c0b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561984541f80 0x5619845536b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 --2- 172.21.15.12:0/3089146154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56198453da80 0x561984553170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 -- 172.21.15.12:0/3089146154 >> 172.21.15.12:0/3089146154 conn(0x56198450aa70 msgr2=0x561984518a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24867fc640 0 client.4674 ms_handle_reset on v2:172.21.15.12:6832/4173401303 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 -- 172.21.15.12:0/3089146154 shutdown_connections 2024-04-27T07:04:33.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 1 -- 172.21.15.12:0/3089146154 wait complete. 2024-04-27T07:04:33.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24b1018000 20 client.4674 trim_cache size 0 max 16384 2024-04-27T07:04:33.480 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:04:33.480 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "e023690c-eecf-4ce8-af7e-ab331203966f", 2024-04-27T07:04:33.480 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-04-27T07:04:33.480 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4242, 2024-04-27T07:04:33.480 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-27T07:04:33.480 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-27T07:04:33.480 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-27T07:04:33.480 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 17.872472645999999, 2024-04-27T07:04:33.480 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 8, 2024-04-27T07:04:33.481 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 20, 2024-04-27T07:04:33.481 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 20, 2024-04-27T07:04:33.481 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 24.942085202000001 2024-04-27T07:04:33.481 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:04:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 -- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f24940683b0 msgr2=0x7f249406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f24940683b0 0x7f249406a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f24a80b7120 tx=0x7f249c009040 comp rx=0 tx=0).stop 2024-04-27T07:04:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 -- 172.21.15.12:0/2446781596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a80ae360 msgr2=0x7f24a81e06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a80ae360 0x7f24a81e06f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2498010900 tx=0x7f2498010dc0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 -- 172.21.15.12:0/2446781596 shutdown_connections 2024-04-27T07:04:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f24940683b0 0x7f249406a870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24a8148f70 0x7f24a81493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f24a8072ab0 0x7f24a81e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 --2- 172.21.15.12:0/2446781596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a80ae360 0x7f24a81e06f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 -- 172.21.15.12:0/2446781596 >> 172.21.15.12:0/2446781596 conn(0x7f24a80b67b0 msgr2=0x7f24a8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:33.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 -- 172.21.15.12:0/2446781596 shutdown_connections 2024-04-27T07:04:33.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.469+0000 7f24ade97640 1 -- 172.21.15.12:0/2446781596 wait complete. 2024-04-27T07:04:33.490 INFO:teuthology.run_tasks:Running task sequential... 2024-04-27T07:04:33.503 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-27T07:04:33.503 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-27T07:04:33.503 INFO:teuthology.task.print:Enabling mgr modules 2024-04-27T07:04:33.503 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-27T07:04:33.535 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-27T07:04:33.535 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-27T07:04:33.536 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-27T07:04:33.536 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-27T07:04:33.536 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-27T07:04:33.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:33.536 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-27T07:04:33.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:33.542 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-04-27T07:04:33.599 INFO:teuthology.orchestra.run.smithi012.stderr:Cannot find device "ceph-brx" 2024-04-27T07:04:33.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:04:33.600 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-27T07:04:33.600 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-27T07:04:33.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.769+0000 7f9294fe6640 1 Processor -- start 2024-04-27T07:04:33.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.769+0000 7f9294fe6640 1 -- start start 2024-04-27T07:04:33.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f9294fe6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9290148370 0x7f92900714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f9294fe6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9290073760 con 0x7f9290073380 2024-04-27T07:04:33.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f9294fe6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f929006e010 con 0x7f9290141770 2024-04-27T07:04:33.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f9294fe6640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f92900719f0 con 0x7f9290148370 2024-04-27T07:04:33.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928e59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9290073380 0x7f9290147c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59808/0 (socket says 172.21.15.12:59808) 2024-04-27T07:04:33.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928e59c640 1 -- 172.21.15.12:0/4019222699 learned_addr learned my addr 172.21.15.12:0/4019222699 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:33.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928dd9b640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9290148370 0x7f92900714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112783475 0 0) 0x7f9290073760 con 0x7f9290073380 2024-04-27T07:04:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f927c003530 con 0x7f9290073380 2024-04-27T07:04:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3939638477 0 0) 0x7f927c003530 con 0x7f9290073380 2024-04-27T07:04:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9290148370 msgr2=0x7f92900714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9290148370 0x7f92900714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 >> v1:172.21.15.53:6789/0 conn(0x7f9290141770 legacy=0x7f9290141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92900b0df0 con 0x7f9290073380 2024-04-27T07:04:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f9278001680 con 0x7f9290073380 2024-04-27T07:04:33.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 >> v1:172.21.15.12:6789/0 conn(0x7f9290073380 legacy=0x7f9290147c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f927c0039f0 0x7f927c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c004350 0x7f927c004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f927c005030 0x7f927c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f92900719f0 con 0x7f927c005030 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f927c00bf40 con 0x7f927c004350 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f927c00c140 con 0x7f927c0039f0 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928ed9d640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f927c005030 0x7f927c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928e59c640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f927c0039f0 0x7f927c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928dd9b640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c004350 0x7f927c004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928ed9d640 1 -- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f927c0039f0 msgr2=0x7f927c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928ed9d640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f927c0039f0 0x7f927c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928ed9d640 1 -- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c004350 msgr2=0x7f927c004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928ed9d640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c004350 0x7f927c004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928ed9d640 1 -- 172.21.15.12:0/4019222699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c003530 con 0x7f927c005030 2024-04-27T07:04:33.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.773+0000 7f928e59c640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f927c0039f0 0x7f927c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:33.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f928ed9d640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f927c005030 0x7f927c00b9d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9288001780 tx=0x7f928801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5040e8006d5002 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92880013a0 con 0x7f927c005030 2024-04-27T07:04:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92880a9070 con 0x7f927c005030 2024-04-27T07:04:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f928d59a640 1 -- 172.21.15.12:0/4019222699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288003600 con 0x7f927c005030 2024-04-27T07:04:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 -- 172.21.15.12:0/4019222699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f927c005030 msgr2=0x7f927c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 --2- 172.21.15.12:0/4019222699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f927c005030 0x7f927c00b9d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9288001780 tx=0x7f928801c420 comp rx=0 tx=0).stop 2024-04-27T07:04:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f928ed9d640 1 -- 172.21.15.12:0/4019222699 reap_dead start 2024-04-27T07:04:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 -- 172.21.15.12:0/4019222699 shutdown_connections 2024-04-27T07:04:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 -- 172.21.15.12:0/4019222699 >> 172.21.15.12:0/4019222699 conn(0x7f92900b9e80 msgr2=0x7f92900ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 -- 172.21.15.12:0/4019222699 shutdown_connections 2024-04-27T07:04:33.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 -- 172.21.15.12:0/4019222699 wait complete. 2024-04-27T07:04:33.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 Processor -- start 2024-04-27T07:04:33.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 -- start start 2024-04-27T07:04:33.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c005030 0x7f92900af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92900b0210 0x7f92900af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f92901d1930 0x7f92901d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9290071740 con 0x7f92900b0210 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f929006ddc0 con 0x7f927c005030 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.777+0000 7f9294fe6640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f92901d14a0 con 0x7f92901d1930 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92900b0210 0x7f92900af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92900b0210 0x7f92900af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55900/0 (socket says 172.21.15.12:55900) 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928dd9b640 1 -- 172.21.15.12:0/2495910291 learned_addr learned my addr 172.21.15.12:0/2495910291 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928e59c640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c005030 0x7f92900af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928ed9d640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f92901d1930 0x7f92901d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928dd9b640 1 -- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f92901d1930 msgr2=0x7f92901d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928dd9b640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f92901d1930 0x7f92901d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928dd9b640 1 -- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c005030 msgr2=0x7f92900af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928dd9b640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c005030 0x7f92900af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928dd9b640 1 -- 172.21.15.12:0/2495910291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92901425f0 con 0x7f92900b0210 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928e59c640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c005030 0x7f92900af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928dd9b640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92900b0210 0x7f92900af540 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9280009f50 tx=0x7f92800115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:33.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f9286ffd640 1 -- 172.21.15.12:0/2495910291 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92800039a0 con 0x7f92900b0210 2024-04-27T07:04:33.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f9286ffd640 1 -- 172.21.15.12:0/2495910291 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928001b070 con 0x7f92900b0210 2024-04-27T07:04:33.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f9294fe6640 1 -- 172.21.15.12:0/2495910291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92901d22f0 con 0x7f92900b0210 2024-04-27T07:04:33.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f9286ffd640 1 -- 172.21.15.12:0/2495910291 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280003b40 con 0x7f92900b0210 2024-04-27T07:04:33.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f9294fe6640 1 -- 172.21.15.12:0/2495910291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92901d1130 con 0x7f92900b0210 2024-04-27T07:04:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f9294fe6640 1 -- 172.21.15.12:0/2495910291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9290069330 con 0x7f92900b0210 2024-04-27T07:04:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f9286ffd640 1 -- 172.21.15.12:0/2495910291 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9280003ce0 con 0x7f92900b0210 2024-04-27T07:04:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f9286ffd640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9270068380 0x7f927006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f9286ffd640 1 -- 172.21.15.12:0/2495910291 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9280087530 con 0x7f92900b0210 2024-04-27T07:04:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928e59c640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9270068380 0x7f927006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.781+0000 7f928e59c640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9270068380 0x7f927006a840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9278001c00 tx=0x7f9278001b90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:33.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.785+0000 7f9286ffd640 1 -- 172.21.15.12:0/2495910291 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9280025020 con 0x7f92900b0210 2024-04-27T07:04:33.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.981+0000 7f9294fe6640 1 -- 172.21.15.12:0/2495910291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f92900b9860 con 0x7f92900b0210 2024-04-27T07:04:33.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9286ffd640 1 -- 172.21.15.12:0/2495910291 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f92901d22f0 con 0x7f92900b0210 2024-04-27T07:04:33.988 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-27T07:04:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 -- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9270068380 msgr2=0x7f927006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9270068380 0x7f927006a840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9278001c00 tx=0x7f9278001b90 comp rx=0 tx=0).stop 2024-04-27T07:04:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 -- 172.21.15.12:0/2495910291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92900b0210 msgr2=0x7f92900af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92900b0210 0x7f92900af540 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9280009f50 tx=0x7f92800115a0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 -- 172.21.15.12:0/2495910291 shutdown_connections 2024-04-27T07:04:33.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9270068380 0x7f927006a840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f92901d1930 0x7f92901d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92900b0210 0x7f92900af540 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 --2- 172.21.15.12:0/2495910291 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f927c005030 0x7f92900af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 -- 172.21.15.12:0/2495910291 >> 172.21.15.12:0/2495910291 conn(0x7f92900b9e80 msgr2=0x7f9290140170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 -- 172.21.15.12:0/2495910291 shutdown_connections 2024-04-27T07:04:33.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:33.985+0000 7f9294fe6640 1 -- 172.21.15.12:0/2495910291 wait complete. 2024-04-27T07:04:34.005 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-27T07:04:34.006 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-27T07:04:34.006 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-04-27T07:04:34.006 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-27T07:04:34.006 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-27T07:04:34.006 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-27T07:04:34.006 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-27T07:04:34.006 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-27T07:04:34.006 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-27T07:04:34.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:34.006 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:04:34.013 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:04:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-04-27T07:04:34.014 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-27T07:04:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-04-27T07:04:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-04-27T07:04:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-27T07:04:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42468sec preferred_lft 42468sec 2024-04-27T07:04:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2024-04-27T07:04:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:04:34.014 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:04:34.014 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-04-27T07:04:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-04-27T07:04:34.015 INFO:teuthology.orchestra.run.smithi012.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-27T07:04:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 02:42:2a:2f:06:4a brd ff:ff:ff:ff:ff:ff 2024-04-27T07:04:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-27T07:04:34.015 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:04:34.015 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-27T07:04:34.016 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:04:34.016 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-27T07:04:34.016 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-04-27T07:04:34.016 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-04-27T07:04:34.016 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-04-27T07:04:34.016 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-27T07:04:34.016 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-27T07:04:34.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:04:34.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:04:34.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:34.138 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-27T07:04:34.192 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-04-27T07:04:34.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:34.194 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-04-27T07:04:34.252 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-04-27T07:04:34.252 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-27T07:04:34.252 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-27T07:04:34.252 INFO:teuthology.orchestra.run.smithi012.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-27T07:04:34.252 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-27T07:04:34.252 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-27T07:04:34.253 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:04:34.253 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-27T07:04:34.253 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-27T07:04:34.253 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-27T07:04:34.253 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-27T07:04:34.253 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-27T07:04:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:04:34.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:04:34.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:34.375 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-27T07:04:34.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:34.382 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-04-27T07:04:34.429 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:04:34.429 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-27T07:04:34.429 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:04:34.429 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-27T07:04:34.429 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-27T07:04:34.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:04:34.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:04:34.549 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-27T07:04:34.549 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:04:34.549 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-27T07:04:34.549 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-27T07:04:34.549 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-27T07:04:34.550 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-27T07:04:34.550 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-27T07:04:34.550 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-27T07:04:34.550 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-27T07:04:34.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:04:34.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:04:34.658 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:04:34.658 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-27T07:04:34.658 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-04-27T07:04:34.658 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-04-27T07:04:34.658 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-27T07:04:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:04:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:04:34.761 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-27T07:04:34.761 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:04:34.761 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-27T07:04:34.808 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-27T07:04:34.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:04:34.808 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-27T07:04:34.855 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-04-27T07:04:34.910 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:04:34.957 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-04-27T07:04:34.957 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-04-27T07:04:34.957 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-04-27T07:04:34.957 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-04-27T07:04:34.957 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/run/lock 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-04-27T07:04:34.958 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/fs/nfsd 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/run/rpc_pipefs 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:04:34.959 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:04:34.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:34.960 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-04-27T07:04:35.009 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-27T07:04:35.009 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-27T07:04:35.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-04-27T07:04:35.069 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:04:35.117 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-04-27T07:04:35.117 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-04-27T07:04:35.117 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-04-27T07:04:35.117 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/run/lock 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-04-27T07:04:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/fs/nfsd 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/run/rpc_pipefs 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:04:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:04:35.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:35.120 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-04-27T07:04:35.136 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:2024-04-27T07:04:35.129+0000 7fb64acf6580 -1 init, newargv = 0x563a4fdfcaa0 newargc=15 2024-04-27T07:04:35.136 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:ceph-fuse[34856]: starting ceph client 2024-04-27T07:04:35.143 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:ceph-fuse[34856]: starting fuse 2024-04-27T07:04:36.242 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-04-27T07:04:36.259 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/run/lock 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-04-27T07:04:36.306 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/fs/nfsd 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/run/rpc_pipefs 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:04:36.307 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:04:36.308 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:04:36.308 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-04-27T07:04:36.308 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:04:36.308 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:04:36.308 INFO:teuthology.orchestra.run.smithi012.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-27T07:04:36.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:36.309 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-04-27T07:04:36.361 INFO:teuthology.orchestra.run.smithi012.stdout:45 2024-04-27T07:04:36.361 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-27T07:04:36.361 DEBUG:teuthology.orchestra.run.smithi012:> sudo stdin-killer -- python3 -c ' 2024-04-27T07:04:36.361 DEBUG:teuthology.orchestra.run.smithi012:> import glob 2024-04-27T07:04:36.361 DEBUG:teuthology.orchestra.run.smithi012:> import re 2024-04-27T07:04:36.361 DEBUG:teuthology.orchestra.run.smithi012:> import os 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> import subprocess 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> def _find_admin_socket(client_name): 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> files = glob.glob(asok_path) 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> # Given a non-glob path, it better be there 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> if "*" not in asok_path: 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> assert(len(files) == 1) 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> return files[0] 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> for f in files: 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-27T07:04:36.362 DEBUG:teuthology.orchestra.run.smithi012:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-27T07:04:36.363 DEBUG:teuthology.orchestra.run.smithi012:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-27T07:04:36.363 DEBUG:teuthology.orchestra.run.smithi012:> contents = proc_f.read() 2024-04-27T07:04:36.363 DEBUG:teuthology.orchestra.run.smithi012:> if mountpoint in contents: 2024-04-27T07:04:36.363 DEBUG:teuthology.orchestra.run.smithi012:> return f 2024-04-27T07:04:36.363 DEBUG:teuthology.orchestra.run.smithi012:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-27T07:04:36.363 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-27T07:04:36.363 DEBUG:teuthology.orchestra.run.smithi012:> print(_find_admin_socket("client.0")) 2024-04-27T07:04:36.363 DEBUG:teuthology.orchestra.run.smithi012:> ' 2024-04-27T07:04:36.488 INFO:teuthology.orchestra.run.smithi012.stdout:/var/run/ceph/ceph-client.0.34856.asok 2024-04-27T07:04:36.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:04:36.500 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34856.asok 2024-04-27T07:04:36.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:36.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34856.asok status 2024-04-27T07:04:36.660 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:04:36.660 INFO:teuthology.orchestra.run.smithi012.stdout: "metadata": { 2024-04-27T07:04:36.660 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_sha1": "c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f", 2024-04-27T07:04:36.660 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-3227-gc37f63b3 (c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f) squid (dev)", 2024-04-27T07:04:36.660 INFO:teuthology.orchestra.run.smithi012.stdout: "entity_id": "0", 2024-04-27T07:04:36.660 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": "smithi012", 2024-04-27T07:04:36.660 INFO:teuthology.orchestra.run.smithi012.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-27T07:04:36.660 INFO:teuthology.orchestra.run.smithi012.stdout: "pid": "34856", 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "root": "/" 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_count": 0, 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_pinned_count": 0, 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4689, 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "inst": { 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": { 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "client", 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "num": 4689 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1387122994 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:36.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1387122994 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "inst_str": "client.4689 192.168.0.1:0/1387122994", 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "addr_str": "192.168.0.1:0/1387122994", 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "inode_count": 1, 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_epoch": 8, 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch": 20, 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch_barrier": 0, 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklisted": false, 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs" 2024-04-27T07:04:36.662 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:04:36.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:36.675 DEBUG:teuthology.orchestra.run.smithi012:> stat --file-system '--printf=%T 2024-04-27T07:04:36.675 DEBUG:teuthology.orchestra.run.smithi012:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-27T07:04:36.682 INFO:teuthology.orchestra.run.smithi012.stdout:fuseblk 2024-04-27T07:04:36.683 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-27T07:04:36.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:36.683 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-27T07:04:36.737 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-27T07:04:36.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:36.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82d20bf640 1 Processor -- start 2024-04-27T07:04:36.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82d20bf640 1 -- start start 2024-04-27T07:04:36.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82d20bf640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82cc0b9a10 0x7f82cc0b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:36.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82d20bf640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f82cc0b3680 0x7f82cc145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:36.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82d20bf640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82cc06dac0 con 0x7f82cc1460b0 2024-04-27T07:04:36.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82d20bf640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f82cc06e890 con 0x7f82cc0b3680 2024-04-27T07:04:36.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82d20bf640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f82cc0b9df0 con 0x7f82cc0b9a10 2024-04-27T07:04:36.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82c3fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f82cc0b3680 0x7f82cc145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:36.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82c3fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f82cc0b3680 0x7f82cc145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55904/0 (socket says 172.21.15.12:55904) 2024-04-27T07:04:36.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82c3fff640 1 -- 172.21.15.12:0/443365005 learned_addr learned my addr 172.21.15.12:0/443365005 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:36.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82cb7fe640 1 --2- 172.21.15.12:0/443365005 >> v2:172.21.15.53:3300/0 conn(0x7f82cc0b9a10 0x7f82cc0b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:36.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82c3fff640 1 -- 172.21.15.12:0/443365005 >> v2:172.21.15.53:3300/0 conn(0x7f82cc0b9a10 msgr2=0x7f82cc0b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:36.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82c3fff640 1 --2- 172.21.15.12:0/443365005 >> v2:172.21.15.53:3300/0 conn(0x7f82cc0b9a10 0x7f82cc0b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:36.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82c3fff640 1 -- 172.21.15.12:0/443365005 >> v1:172.21.15.53:6789/0 conn(0x7f82cc1460b0 legacy=0x7f82cc1490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:36.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82c3fff640 1 -- 172.21.15.12:0/443365005 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc06e090 con 0x7f82cc0b3680 2024-04-27T07:04:36.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.905+0000 7f82c3fff640 1 --2- 172.21.15.12:0/443365005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc0b3680 0x7f82cc145b70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f82cc06ed00 tx=0x7f82b401c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c09d75344096e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:36.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82caffd640 1 -- 172.21.15.12:0/443365005 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b40b1070 con 0x7f82cc0b3680 2024-04-27T07:04:36.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82caffd640 1 -- 172.21.15.12:0/443365005 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82b40ac440 con 0x7f82cc0b3680 2024-04-27T07:04:36.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82caffd640 1 -- 172.21.15.12:0/443365005 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b40ac810 con 0x7f82cc0b3680 2024-04-27T07:04:36.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- 172.21.15.12:0/443365005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc0b3680 msgr2=0x7f82cc145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:36.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 --2- 172.21.15.12:0/443365005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc0b3680 0x7f82cc145b70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f82cc06ed00 tx=0x7f82b401c1e0 comp rx=0 tx=0).stop 2024-04-27T07:04:36.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- 172.21.15.12:0/443365005 shutdown_connections 2024-04-27T07:04:36.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 --2- 172.21.15.12:0/443365005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc0b3680 0x7f82cc145b70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:36.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 --2- 172.21.15.12:0/443365005 >> v2:172.21.15.53:3300/0 conn(0x7f82cc0b9a10 0x7f82cc0b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:36.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- 172.21.15.12:0/443365005 >> 172.21.15.12:0/443365005 conn(0x7f82cc067d10 msgr2=0x7f82cc068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:36.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- 172.21.15.12:0/443365005 shutdown_connections 2024-04-27T07:04:36.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- 172.21.15.12:0/443365005 wait complete. 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 Processor -- start 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- start start 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82cc0b3680 0x7f82cc13d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82cc0b9a10 0x7f82cc147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc1460b0 0x7f82cc148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc0b6c80 con 0x7f82cc1460b0 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc0b6b00 con 0x7f82cc0b9a10 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f82cc153430 con 0x7f82cc0b3680 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc1460b0 0x7f82cc148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc1460b0 0x7f82cc148410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55912/0 (socket says 172.21.15.12:55912) 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cbfff640 1 -- 172.21.15.12:0/303587564 learned_addr learned my addr 172.21.15.12:0/303587564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82c3fff640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82cc0b9a10 0x7f82cc147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:36.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cb7fe640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82cc0b3680 0x7f82cc13d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:36.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cbfff640 1 -- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82cc0b3680 msgr2=0x7f82cc13d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:36.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cbfff640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82cc0b3680 0x7f82cc13d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:36.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cbfff640 1 -- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82cc0b9a10 msgr2=0x7f82cc147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:36.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cbfff640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82cc0b9a10 0x7f82cc147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:36.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cbfff640 1 -- 172.21.15.12:0/303587564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc0b1e10 con 0x7f82cc1460b0 2024-04-27T07:04:36.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82c3fff640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82cc0b9a10 0x7f82cc147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:36.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cb7fe640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82cc0b3680 0x7f82cc13d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:36.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82cbfff640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc1460b0 0x7f82cc148410 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f82c400c7b0 tx=0x7f82c400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:36.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82c8ff9640 1 -- 172.21.15.12:0/303587564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4004b70 con 0x7f82cc1460b0 2024-04-27T07:04:36.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82c8ff9640 1 -- 172.21.15.12:0/303587564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c4004d10 con 0x7f82cc1460b0 2024-04-27T07:04:36.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- 172.21.15.12:0/303587564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82cc0b20f0 con 0x7f82cc1460b0 2024-04-27T07:04:36.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82c8ff9640 1 -- 172.21.15.12:0/303587564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c400aa90 con 0x7f82cc1460b0 2024-04-27T07:04:36.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.909+0000 7f82d20bf640 1 -- 172.21.15.12:0/303587564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82cc0b8750 con 0x7f82cc1460b0 2024-04-27T07:04:36.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.913+0000 7f82d20bf640 1 -- 172.21.15.12:0/303587564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82cc151080 con 0x7f82cc1460b0 2024-04-27T07:04:36.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.913+0000 7f82c8ff9640 1 -- 172.21.15.12:0/303587564 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f82c400ac30 con 0x7f82cc1460b0 2024-04-27T07:04:36.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.913+0000 7f82c8ff9640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f82ac0683b0 0x7f82ac06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:36.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.913+0000 7f82c8ff9640 1 -- 172.21.15.12:0/303587564 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f82c4086c50 con 0x7f82cc1460b0 2024-04-27T07:04:36.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.913+0000 7f82cb7fe640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f82ac0683b0 0x7f82ac06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:36.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.913+0000 7f82cb7fe640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f82ac0683b0 0x7f82ac06a870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009840 tx=0x7f82bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:36.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:36.917+0000 7f82c8ff9640 1 -- 172.21.15.12:0/303587564 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f82cc151080 con 0x7f82cc1460b0 2024-04-27T07:04:37.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.081+0000 7f82d20bf640 1 -- 172.21.15.12:0/303587564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82cc068c20 con 0x7f82cc1460b0 2024-04-27T07:04:37.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82c8ff9640 1 -- 172.21.15.12:0/303587564 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f82c4020070 con 0x7f82cc1460b0 2024-04-27T07:04:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:37.089 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:37.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 -- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f82ac0683b0 msgr2=0x7f82ac06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f82ac0683b0 0x7f82ac06a870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009840 tx=0x7f82bc008040 comp rx=0 tx=0).stop 2024-04-27T07:04:37.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 -- 172.21.15.12:0/303587564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc1460b0 msgr2=0x7f82cc148410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc1460b0 0x7f82cc148410 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f82c400c7b0 tx=0x7f82c400cc70 comp rx=0 tx=0).stop 2024-04-27T07:04:37.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 -- 172.21.15.12:0/303587564 shutdown_connections 2024-04-27T07:04:37.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f82ac0683b0 0x7f82ac06a870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82cc1460b0 0x7f82cc148410 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82cc0b9a10 0x7f82cc147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 --2- 172.21.15.12:0/303587564 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f82cc0b3680 0x7f82cc13d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 -- 172.21.15.12:0/303587564 >> 172.21.15.12:0/303587564 conn(0x7f82cc067d10 msgr2=0x7f82cc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:37.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 -- 172.21.15.12:0/303587564 shutdown_connections 2024-04-27T07:04:37.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.085+0000 7f82d20bf640 1 -- 172.21.15.12:0/303587564 wait complete. 2024-04-27T07:04:37.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:12.742740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:13.912898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:37.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:37.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0f0d0640 1 Processor -- start 2024-04-27T07:04:37.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0f0d0640 1 -- start start 2024-04-27T07:04:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0f0d0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3d080b5ae0 0x7f3d08142010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0f0d0640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d080b6860 con 0x7f3d080b6480 2024-04-27T07:04:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0f0d0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d08147ee0 con 0x7f3d081489c0 2024-04-27T07:04:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0f0d0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3d081481c0 con 0x7f3d080b5ae0 2024-04-27T07:04:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0ce6c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3d080b6480 0x7f3d08141900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59822/0 (socket says 172.21.15.12:59822) 2024-04-27T07:04:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0ce6c640 1 -- 172.21.15.12:0/2546851128 learned_addr learned my addr 172.21.15.12:0/2546851128 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0d66d640 1 --2- 172.21.15.12:0/2546851128 >> v2:172.21.15.53:3300/0 conn(0x7f3d080b5ae0 0x7f3d08142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 252363047 0 0) 0x7f3d080b6860 con 0x7f3d080b6480 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cf8003530 con 0x7f3d080b6480 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2424030713 0 0) 0x7f3cf8003530 con 0x7f3d080b6480 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 >> v2:172.21.15.53:3300/0 conn(0x7f3d080b5ae0 msgr2=0x7f3d08142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 --2- 172.21.15.12:0/2546851128 >> v2:172.21.15.53:3300/0 conn(0x7f3d080b5ae0 0x7f3d08142010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 >> v1:172.21.15.53:6789/0 conn(0x7f3d081489c0 legacy=0x7f3d08150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d081484a0 con 0x7f3d080b6480 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f3cfc001680 con 0x7f3d080b6480 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 >> v1:172.21.15.12:6789/0 conn(0x7f3d080b6480 legacy=0x7f3d08141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3cf80039c0 0x7f3cf8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cf8004810 0x7f3cf8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf80051d0 0x7f3cf800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081481c0 con 0x7f3cf8004810 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf800c170 con 0x7f3cf80051d0 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf800c370 con 0x7f3cf80039c0 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d07fff640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cf8004810 0x7f3cf8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0ce6c640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3cf80039c0 0x7f3cf8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0d66d640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf80051d0 0x7f3cf800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d07fff640 1 -- 172.21.15.12:0/2546851128 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3cf80039c0 msgr2=0x7f3cf8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d07fff640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3cf80039c0 0x7f3cf8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d07fff640 1 -- 172.21.15.12:0/2546851128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf80051d0 msgr2=0x7f3cf800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d07fff640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3cf80051d0 0x7f3cf800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d07fff640 1 -- 172.21.15.12:0/2546851128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf8003530 con 0x7f3cf8004810 2024-04-27T07:04:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.225+0000 7f3d0d66d640 1 -- 172.21.15.12:0/2546851128 reap_dead start 2024-04-27T07:04:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d07fff640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cf8004810 0x7f3cf8004c90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4001780 tx=0x7f3cf401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8035626960cc7c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:37.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40013a0 con 0x7f3cf8004810 2024-04-27T07:04:37.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf40052f0 con 0x7f3cf8004810 2024-04-27T07:04:37.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d077fe640 1 -- 172.21.15.12:0/2546851128 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4003840 con 0x7f3cf8004810 2024-04-27T07:04:37.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/2546851128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cf8004810 msgr2=0x7f3cf8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d0f0d0640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cf8004810 0x7f3cf8004c90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4001780 tx=0x7f3cf401c400 comp rx=0 tx=0).stop 2024-04-27T07:04:37.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/2546851128 shutdown_connections 2024-04-27T07:04:37.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d0f0d0640 1 --2- 172.21.15.12:0/2546851128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cf8004810 0x7f3cf8004c90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/2546851128 >> 172.21.15.12:0/2546851128 conn(0x7f3d080bd620 msgr2=0x7f3d080bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:37.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/2546851128 shutdown_connections 2024-04-27T07:04:37.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/2546851128 wait complete. 2024-04-27T07:04:37.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d0f0d0640 1 Processor -- start 2024-04-27T07:04:37.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.229+0000 7f3d0f0d0640 1 -- start start 2024-04-27T07:04:37.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0f0d0640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3cf8004810 0x7f3d08144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0f0d0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0814a540 0x7f3d0814a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0f0d0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d081e3c00 0x7f3d081e4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0f0d0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08142110 con 0x7f3d0814a540 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0f0d0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3d080b5ec0 con 0x7f3d081e3c00 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0f0d0640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3d08142290 con 0x7f3cf8004810 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0814a540 0x7f3d0814a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0814a540 0x7f3d0814a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55930/0 (socket says 172.21.15.12:55930) 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d07fff640 1 -- 172.21.15.12:0/1383007746 learned_addr learned my addr 172.21.15.12:0/1383007746 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0ce6c640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3cf8004810 0x7f3d08144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0d66d640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d081e3c00 0x7f3d081e4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d07fff640 1 -- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3cf8004810 msgr2=0x7f3d08144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d07fff640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3cf8004810 0x7f3d08144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d07fff640 1 -- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d081e3c00 msgr2=0x7f3d081e4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d07fff640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d081e3c00 0x7f3d081e4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d07fff640 1 -- 172.21.15.12:0/1383007746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d081d3a10 con 0x7f3d0814a540 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0d66d640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d081e3c00 0x7f3d081e4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d07fff640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0814a540 0x7f3d0814a9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3cf401c230 tx=0x7f3cf8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:37.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d057fa640 1 -- 172.21.15.12:0/1383007746 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40b8020 con 0x7f3d0814a540 2024-04-27T07:04:37.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d057fa640 1 -- 172.21.15.12:0/1383007746 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf40054a0 con 0x7f3d0814a540 2024-04-27T07:04:37.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/1383007746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0814af00 con 0x7f3d0814a540 2024-04-27T07:04:37.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d057fa640 1 -- 172.21.15.12:0/1383007746 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4007a40 con 0x7f3d0814a540 2024-04-27T07:04:37.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/1383007746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0814b300 con 0x7f3d0814a540 2024-04-27T07:04:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/1383007746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d08068c20 con 0x7f3d0814a540 2024-04-27T07:04:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d057fa640 1 -- 172.21.15.12:0/1383007746 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3cf40bb020 con 0x7f3d0814a540 2024-04-27T07:04:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d057fa640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cec0683b0 0x7f3cec06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d057fa640 1 -- 172.21.15.12:0/1383007746 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3cf4120820 con 0x7f3d0814a540 2024-04-27T07:04:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0ce6c640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cec0683b0 0x7f3cec06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.233+0000 7f3d0ce6c640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cec0683b0 0x7f3cec06a870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c10 tx=0x7f3cfc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:37.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.237+0000 7f3d057fa640 1 -- 172.21.15.12:0/1383007746 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3cf40c2030 con 0x7f3d0814a540 2024-04-27T07:04:37.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/1383007746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3d081451a0 con 0x7f3d0814a540 2024-04-27T07:04:37.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d057fa640 1 -- 172.21.15.12:0/1383007746 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f3d0814af00 con 0x7f3d0814a540 2024-04-27T07:04:37.402 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cec0683b0 msgr2=0x7f3cec06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cec0683b0 0x7f3cec06a870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c10 tx=0x7f3cfc009040 comp rx=0 tx=0).stop 2024-04-27T07:04:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/1383007746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0814a540 msgr2=0x7f3d0814a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0814a540 0x7f3d0814a9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3cf401c230 tx=0x7f3cf8005840 comp rx=0 tx=0).stop 2024-04-27T07:04:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/1383007746 shutdown_connections 2024-04-27T07:04:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3cec0683b0 0x7f3cec06a870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d081e3c00 0x7f3d081e4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0814a540 0x7f3d0814a9c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 --2- 172.21.15.12:0/1383007746 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3cf8004810 0x7f3d08144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/1383007746 >> 172.21.15.12:0/1383007746 conn(0x7f3d080bd620 msgr2=0x7f3d08074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:37.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/1383007746 shutdown_connections 2024-04-27T07:04:37.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.397+0000 7f3d0f0d0640 1 -- 172.21.15.12:0/1383007746 wait complete. 2024-04-27T07:04:37.419 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:04:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.529+0000 7f7e2b837640 1 Processor -- start 2024-04-27T07:04:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.529+0000 7f7e2b837640 1 -- start start 2024-04-27T07:04:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e2b837640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7e240b3650 0x7f7e240b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e2b837640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e2406d300 0x7f7e240b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e2b837640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e240b4d70 con 0x7f7e2406e370 2024-04-27T07:04:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e2b837640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7e240b5c10 con 0x7f7e240b3650 2024-04-27T07:04:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e2b837640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7e240740d0 con 0x7f7e2406d300 2024-04-27T07:04:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e295d3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7e2406e370 0x7f7e2413faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43076/0 (socket says 172.21.15.12:43076) 2024-04-27T07:04:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e29dd4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e2406d300 0x7f7e240b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e295d3640 1 -- 172.21.15.12:0/3615206968 learned_addr learned my addr 172.21.15.12:0/3615206968 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e28dd2640 1 --2- 172.21.15.12:0/3615206968 >> v2:172.21.15.53:3300/0 conn(0x7f7e240b3650 0x7f7e240b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4282016409 0 0) 0x7f7e240b4d70 con 0x7f7e2406e370 2024-04-27T07:04:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e10003530 con 0x7f7e2406e370 2024-04-27T07:04:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2704511068 0 0) 0x7f7e10003530 con 0x7f7e2406e370 2024-04-27T07:04:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e2406d300 msgr2=0x7f7e240b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e2406d300 0x7f7e240b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 >> v2:172.21.15.53:3300/0 conn(0x7f7e240b3650 msgr2=0x7f7e240b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 --2- 172.21.15.12:0/3615206968 >> v2:172.21.15.53:3300/0 conn(0x7f7e240b3650 0x7f7e240b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e24073e60 con 0x7f7e2406e370 2024-04-27T07:04:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f7e18001680 con 0x7f7e2406e370 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 >> v1:172.21.15.12:6789/0 conn(0x7f7e2406e370 legacy=0x7f7e2413faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e100039c0 0x7f7e10003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e10004810 0x7f7e10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e2406d300 0x7f7e100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7e240b5c10 con 0x7f7e10004810 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e240740d0 con 0x7f7e100039c0 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1000b9d0 con 0x7f7e2406d300 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e28dd2640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e10004810 0x7f7e10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e295d3640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e100039c0 0x7f7e10003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e29dd4640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e2406d300 0x7f7e100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e28dd2640 1 -- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e2406d300 msgr2=0x7f7e100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e28dd2640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e2406d300 0x7f7e100051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e28dd2640 1 -- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e100039c0 msgr2=0x7f7e10003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e28dd2640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e100039c0 0x7f7e10003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e28dd2640 1 -- 172.21.15.12:0/3615206968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e10003530 con 0x7f7e10004810 2024-04-27T07:04:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e295d3640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e100039c0 0x7f7e10003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.533+0000 7f7e28dd2640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e10004810 0x7f7e10004c90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009ae0 tx=0x7f7e2001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536f5b9c9a12eceb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e200096b0 con 0x7f7e10004810 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e200a9470 con 0x7f7e10004810 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e17fff640 1 -- 172.21.15.12:0/3615206968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e200a9860 con 0x7f7e10004810 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- 172.21.15.12:0/3615206968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e10004810 msgr2=0x7f7e10004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 --2- 172.21.15.12:0/3615206968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e10004810 0x7f7e10004c90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009ae0 tx=0x7f7e2001ca70 comp rx=0 tx=0).stop 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e29dd4640 1 -- 172.21.15.12:0/3615206968 reap_dead start 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- 172.21.15.12:0/3615206968 shutdown_connections 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- 172.21.15.12:0/3615206968 >> 172.21.15.12:0/3615206968 conn(0x7f7e240b93b0 msgr2=0x7f7e240b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- 172.21.15.12:0/3615206968 shutdown_connections 2024-04-27T07:04:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- 172.21.15.12:0/3615206968 wait complete. 2024-04-27T07:04:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 Processor -- start 2024-04-27T07:04:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- start start 2024-04-27T07:04:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e240b8af0 0x7f7e241e1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e24071240 0x7f7e24138450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24138990 0x7f7e2409d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24150c30 con 0x7f7e24138990 2024-04-27T07:04:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241401b0 con 0x7f7e240b8af0 2024-04-27T07:04:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7e24150db0 con 0x7f7e24071240 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e29dd4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24138990 0x7f7e2409d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e29dd4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24138990 0x7f7e2409d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54664/0 (socket says 172.21.15.12:54664) 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e29dd4640 1 -- 172.21.15.12:0/1220054028 learned_addr learned my addr 172.21.15.12:0/1220054028 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e295d3640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e240b8af0 0x7f7e241e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e28dd2640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e24071240 0x7f7e24138450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e29dd4640 1 -- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e24071240 msgr2=0x7f7e24138450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e29dd4640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e24071240 0x7f7e24138450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e29dd4640 1 -- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e240b8af0 msgr2=0x7f7e241e1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e29dd4640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e240b8af0 0x7f7e241e1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e29dd4640 1 -- 172.21.15.12:0/1220054028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e240b8ed0 con 0x7f7e24138990 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e29dd4640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24138990 0x7f7e2409d270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e2406d730 tx=0x7f7e1c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e15ffb640 1 -- 172.21.15.12:0/1220054028 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c01c070 con 0x7f7e24138990 2024-04-27T07:04:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e15ffb640 1 -- 172.21.15.12:0/1220054028 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1c017400 con 0x7f7e24138990 2024-04-27T07:04:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- 172.21.15.12:0/1220054028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e24072280 con 0x7f7e24138990 2024-04-27T07:04:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e15ffb640 1 -- 172.21.15.12:0/1220054028 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c0177f0 con 0x7f7e24138990 2024-04-27T07:04:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.537+0000 7f7e2b837640 1 -- 172.21.15.12:0/1220054028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e24072840 con 0x7f7e24138990 2024-04-27T07:04:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.541+0000 7f7e2b837640 1 -- 172.21.15.12:0/1220054028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e24068c20 con 0x7f7e24138990 2024-04-27T07:04:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.541+0000 7f7e15ffb640 1 -- 172.21.15.12:0/1220054028 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7e1c01f070 con 0x7f7e24138990 2024-04-27T07:04:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.541+0000 7f7e15ffb640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e080683b0 0x7f7e0806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.541+0000 7f7e15ffb640 1 -- 172.21.15.12:0/1220054028 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e1c086bf0 con 0x7f7e24138990 2024-04-27T07:04:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.541+0000 7f7e295d3640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e080683b0 0x7f7e0806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.541+0000 7f7e295d3640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e080683b0 0x7f7e0806a870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7e18001870 tx=0x7f7e18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:37.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.545+0000 7f7e15ffb640 1 -- 172.21.15.12:0/1220054028 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e1c012070 con 0x7f7e24138990 2024-04-27T07:04:37.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.701+0000 7f7e2b837640 1 -- 172.21.15.12:0/1220054028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e24072a10 con 0x7f7e24138990 2024-04-27T07:04:37.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.701+0000 7f7e15ffb640 1 -- 172.21.15.12:0/1220054028 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7e1c057240 con 0x7f7e24138990 2024-04-27T07:04:37.707 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:04:37.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 -- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e080683b0 msgr2=0x7f7e0806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e080683b0 0x7f7e0806a870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7e18001870 tx=0x7f7e18009040 comp rx=0 tx=0).stop 2024-04-27T07:04:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 -- 172.21.15.12:0/1220054028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24138990 msgr2=0x7f7e2409d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24138990 0x7f7e2409d270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e2406d730 tx=0x7f7e1c00f4a0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 -- 172.21.15.12:0/1220054028 shutdown_connections 2024-04-27T07:04:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7e080683b0 0x7f7e0806a870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e24138990 0x7f7e2409d270 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7e24071240 0x7f7e24138450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 --2- 172.21.15.12:0/1220054028 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7e240b8af0 0x7f7e241e1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 -- 172.21.15.12:0/1220054028 >> 172.21.15.12:0/1220054028 conn(0x7f7e240b93b0 msgr2=0x7f7e2413f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:37.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 -- 172.21.15.12:0/1220054028 shutdown_connections 2024-04-27T07:04:37.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.705+0000 7f7e2b837640 1 -- 172.21.15.12:0/1220054028 wait complete. 2024-04-27T07:04:37.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:04:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc68206d640 1 Processor -- start 2024-04-27T07:04:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc68206d640 1 -- start start 2024-04-27T07:04:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc68206d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc67c0b6610 0x7fc67c1489d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc68206d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c1419d0 0x7fc67c141e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc68206d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc67c0b79b0 con 0x7fc67c0b8020 2024-04-27T07:04:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc68206d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc67c0b69f0 con 0x7fc67c0b6610 2024-04-27T07:04:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc68206d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc67c147ef0 con 0x7fc67c1419d0 2024-04-27T07:04:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc67b7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc67c0b6610 0x7fc67c1489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc67bfff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c1419d0 0x7fc67c141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc67affd640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc67c0b8020 0x7fc67c0b57b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:33080/0 (socket says 172.21.15.12:33080) 2024-04-27T07:04:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc67affd640 1 -- 172.21.15.12:0/821920099 learned_addr learned my addr 172.21.15.12:0/821920099 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc67b7fe640 1 -- 172.21.15.12:0/821920099 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c1419d0 msgr2=0x7fc67c141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc67b7fe640 1 --2- 172.21.15.12:0/821920099 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c1419d0 0x7fc67c141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc67b7fe640 1 -- 172.21.15.12:0/821920099 >> v1:172.21.15.53:6789/0 conn(0x7fc67c0b8020 legacy=0x7fc67c0b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc67a7fc640 1 -- 172.21.15.12:0/821920099 <== mon.1 v1:172.21.15.53:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2346870672 0 0) 0x7fc67c0b79b0 con 0x7fc67c0b8020 2024-04-27T07:04:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc67b7fe640 1 -- 172.21.15.12:0/821920099 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c0b84b0 con 0x7fc67c0b6610 2024-04-27T07:04:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.833+0000 7fc67bfff640 1 --2- 172.21.15.12:0/821920099 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c1419d0 0x7fc67c141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:04:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc67b7fe640 1 --2- 172.21.15.12:0/821920099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc67c0b6610 0x7fc67c1489d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0097c0 tx=0x7fc66c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59539c5d9b30af66 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc67a7fc640 1 -- 172.21.15.12:0/821920099 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0b1070 con 0x7fc67c0b6610 2024-04-27T07:04:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc67a7fc640 1 -- 172.21.15.12:0/821920099 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66c0a8050 con 0x7fc67c0b6610 2024-04-27T07:04:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc67a7fc640 1 -- 172.21.15.12:0/821920099 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0ac860 con 0x7fc67c0b6610 2024-04-27T07:04:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 -- 172.21.15.12:0/821920099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc67c0b6610 msgr2=0x7fc67c1489d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 --2- 172.21.15.12:0/821920099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc67c0b6610 0x7fc67c1489d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0097c0 tx=0x7fc66c01c3e0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 -- 172.21.15.12:0/821920099 shutdown_connections 2024-04-27T07:04:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 --2- 172.21.15.12:0/821920099 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c1419d0 0x7fc67c141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 --2- 172.21.15.12:0/821920099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc67c0b6610 0x7fc67c1489d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 -- 172.21.15.12:0/821920099 >> 172.21.15.12:0/821920099 conn(0x7fc67c0ba140 msgr2=0x7fc67c0ba550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 -- 172.21.15.12:0/821920099 shutdown_connections 2024-04-27T07:04:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 -- 172.21.15.12:0/821920099 wait complete. 2024-04-27T07:04:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 Processor -- start 2024-04-27T07:04:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 -- start start 2024-04-27T07:04:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc67c0b6610 0x7fc67c1345a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c0b8020 0x7fc67c137e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc67c1419d0 0x7fc67c1e3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c0b6040 con 0x7fc67c0b6610 2024-04-27T07:04:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c0b5ec0 con 0x7fc67c1419d0 2024-04-27T07:04:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc68206d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc67c154ef0 con 0x7fc67c0b8020 2024-04-27T07:04:37.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc67b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc67c0b6610 0x7fc67c1345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc67b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc67c0b6610 0x7fc67c1345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54678/0 (socket says 172.21.15.12:54678) 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.837+0000 7fc67b7fe640 1 -- 172.21.15.12:0/1529562343 learned_addr learned my addr 172.21.15.12:0/1529562343 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67affd640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c0b8020 0x7fc67c137e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67bfff640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc67c1419d0 0x7fc67c1e3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67b7fe640 1 -- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c0b8020 msgr2=0x7fc67c137e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67b7fe640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c0b8020 0x7fc67c137e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67b7fe640 1 -- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc67c1419d0 msgr2=0x7fc67c1e3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67b7fe640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc67c1419d0 0x7fc67c1e3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67b7fe640 1 -- 172.21.15.12:0/1529562343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c135ee0 con 0x7fc67c0b6610 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67affd640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c0b8020 0x7fc67c137e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67b7fe640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc67c0b6610 0x7fc67c1345a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0af040 tx=0x7fc66c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:37.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc663fff640 1 -- 172.21.15.12:0/1529562343 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0b1050 con 0x7fc67c0b6610 2024-04-27T07:04:37.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc663fff640 1 -- 172.21.15.12:0/1529562343 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66c0bd070 con 0x7fc67c0b6610 2024-04-27T07:04:37.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc68206d640 1 -- 172.21.15.12:0/1529562343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67c137230 con 0x7fc67c0b6610 2024-04-27T07:04:37.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc663fff640 1 -- 172.21.15.12:0/1529562343 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0b88b0 con 0x7fc67c0b6610 2024-04-27T07:04:37.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc68206d640 1 -- 172.21.15.12:0/1529562343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67c138350 con 0x7fc67c0b6610 2024-04-27T07:04:37.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc68206d640 1 -- 172.21.15.12:0/1529562343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc67c069330 con 0x7fc67c0b6610 2024-04-27T07:04:37.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc663fff640 1 -- 172.21.15.12:0/1529562343 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc66c0b8a50 con 0x7fc67c0b6610 2024-04-27T07:04:37.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc663fff640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc668068380 0x7fc66806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:37.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc663fff640 1 -- 172.21.15.12:0/1529562343 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc66c1211f0 con 0x7fc67c0b6610 2024-04-27T07:04:37.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67affd640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc668068380 0x7fc66806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:37.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.841+0000 7fc67affd640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc668068380 0x7fc66806a840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc67c135230 tx=0x7fc670009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:37.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:37.845+0000 7fc663fff640 1 -- 172.21.15.12:0/1529562343 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc66c016090 con 0x7fc67c0b6610 2024-04-27T07:04:38.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.001+0000 7fc68206d640 1 -- 172.21.15.12:0/1529562343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc67c145770 con 0x7fc67c0b6610 2024-04-27T07:04:38.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.001+0000 7fc663fff640 1 -- 172.21.15.12:0/1529562343 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fc66c0f1960 con 0x7fc67c0b6610 2024-04-27T07:04:38.006 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:04:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 -- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc668068380 msgr2=0x7fc66806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc668068380 0x7fc66806a840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc67c135230 tx=0x7fc670009040 comp rx=0 tx=0).stop 2024-04-27T07:04:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 -- 172.21.15.12:0/1529562343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc67c0b6610 msgr2=0x7fc67c1345a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc67c0b6610 0x7fc67c1345a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0af040 tx=0x7fc66c01ac20 comp rx=0 tx=0).stop 2024-04-27T07:04:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 -- 172.21.15.12:0/1529562343 shutdown_connections 2024-04-27T07:04:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc668068380 0x7fc66806a840 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc67c1419d0 0x7fc67c1e3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc67c0b8020 0x7fc67c137e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 --2- 172.21.15.12:0/1529562343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc67c0b6610 0x7fc67c1345a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 -- 172.21.15.12:0/1529562343 >> 172.21.15.12:0/1529562343 conn(0x7fc67c0ba140 msgr2=0x7fc67c146550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 -- 172.21.15.12:0/1529562343 shutdown_connections 2024-04-27T07:04:38.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.005+0000 7fc68206d640 1 -- 172.21.15.12:0/1529562343 wait complete. 2024-04-27T07:04:38.024 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:38.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e5420c640 1 Processor -- start 2024-04-27T07:04:38.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e5420c640 1 -- start start 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e5420c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1e4c067b70 0x7f1e4c1495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e5420c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4c0b2010 con 0x7f1e4c0b12c0 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e5420c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4c067f50 con 0x7f1e4c1457a0 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e5420c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4c0b6190 con 0x7f1e4c067b70 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e51fa8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1e4c067b70 0x7f1e4c1495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e517a7640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1e4c0b12c0 0x7f1e4c145090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43084/0 (socket says 172.21.15.12:43084) 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e51fa8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1e4c067b70 0x7f1e4c1495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54688/0 (socket says 172.21.15.12:54688) 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e517a7640 1 -- 172.21.15.12:0/3202560559 learned_addr learned my addr 172.21.15.12:0/3202560559 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e51fa8640 1 -- 172.21.15.12:0/3202560559 >> v1:172.21.15.53:6789/0 conn(0x7f1e4c1457a0 legacy=0x7f1e4c150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e51fa8640 1 -- 172.21.15.12:0/3202560559 >> v1:172.21.15.12:6789/0 conn(0x7f1e4c0b12c0 legacy=0x7f1e4c145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e50fa6640 1 -- 172.21.15.12:0/3202560559 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1948723213 0 0) 0x7f1e4c0b2010 con 0x7f1e4c0b12c0 2024-04-27T07:04:38.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.141+0000 7f1e51fa8640 1 -- 172.21.15.12:0/3202560559 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c0b63f0 con 0x7f1e4c067b70 2024-04-27T07:04:38.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e51fa8640 1 --2- 172.21.15.12:0/3202560559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c067b70 0x7f1e4c1495a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009d90 tx=0x7f1e4801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f357ac7db4a0222b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:38.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e50fa6640 1 -- 172.21.15.12:0/3202560559 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e480b2070 con 0x7f1e4c067b70 2024-04-27T07:04:38.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e50fa6640 1 -- 172.21.15.12:0/3202560559 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e480a5760 con 0x7f1e4c067b70 2024-04-27T07:04:38.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e50fa6640 1 -- 172.21.15.12:0/3202560559 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e480a5b30 con 0x7f1e4c067b70 2024-04-27T07:04:38.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- 172.21.15.12:0/3202560559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c067b70 msgr2=0x7f1e4c1495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 --2- 172.21.15.12:0/3202560559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c067b70 0x7f1e4c1495a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009d90 tx=0x7f1e4801c8a0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- 172.21.15.12:0/3202560559 shutdown_connections 2024-04-27T07:04:38.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 --2- 172.21.15.12:0/3202560559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c067b70 0x7f1e4c1495a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- 172.21.15.12:0/3202560559 >> 172.21.15.12:0/3202560559 conn(0x7f1e4c0b68c0 msgr2=0x7f1e4c0b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:38.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- 172.21.15.12:0/3202560559 shutdown_connections 2024-04-27T07:04:38.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- 172.21.15.12:0/3202560559 wait complete. 2024-04-27T07:04:38.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 Processor -- start 2024-04-27T07:04:38.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- start start 2024-04-27T07:04:38.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e4c067b70 0x7f1e4c141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e4c0b12c0 0x7f1e4c0a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c1457a0 0x7f1e4c0a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c154950 con 0x7f1e4c1457a0 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1547d0 con 0x7f1e4c0b12c0 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4c154ad0 con 0x7f1e4c067b70 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e527a9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c1457a0 0x7f1e4c0a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e527a9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c1457a0 0x7f1e4c0a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54704/0 (socket says 172.21.15.12:54704) 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e527a9640 1 -- 172.21.15.12:0/1595716640 learned_addr learned my addr 172.21.15.12:0/1595716640 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e517a7640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e4c0b12c0 0x7f1e4c0a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e51fa8640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e4c067b70 0x7f1e4c141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e527a9640 1 -- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e4c067b70 msgr2=0x7f1e4c141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e527a9640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e4c067b70 0x7f1e4c141ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e527a9640 1 -- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e4c0b12c0 msgr2=0x7f1e4c0a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e527a9640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e4c0b12c0 0x7f1e4c0a9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e527a9640 1 -- 172.21.15.12:0/1595716640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c0ad520 con 0x7f1e4c1457a0 2024-04-27T07:04:38.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e527a9640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c1457a0 0x7f1e4c0a9f40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400b440 tx=0x7f1e4400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:38.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e427fc640 1 -- 172.21.15.12:0/1595716640 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44015020 con 0x7f1e4c1457a0 2024-04-27T07:04:38.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e427fc640 1 -- 172.21.15.12:0/1595716640 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e44011850 con 0x7f1e4c1457a0 2024-04-27T07:04:38.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- 172.21.15.12:0/1595716640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c1426f0 con 0x7f1e4c1457a0 2024-04-27T07:04:38.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e427fc640 1 -- 172.21.15.12:0/1595716640 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44011c40 con 0x7f1e4c1457a0 2024-04-27T07:04:38.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.145+0000 7f1e5420c640 1 -- 172.21.15.12:0/1595716640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c0ac5f0 con 0x7f1e4c1457a0 2024-04-27T07:04:38.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.149+0000 7f1e5420c640 1 -- 172.21.15.12:0/1595716640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c068c20 con 0x7f1e4c1457a0 2024-04-27T07:04:38.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.149+0000 7f1e427fc640 1 -- 172.21.15.12:0/1595716640 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1e44018020 con 0x7f1e4c1457a0 2024-04-27T07:04:38.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.149+0000 7f1e427fc640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e300683b0 0x7f1e3006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.149+0000 7f1e427fc640 1 -- 172.21.15.12:0/1595716640 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e44086890 con 0x7f1e4c1457a0 2024-04-27T07:04:38.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.149+0000 7f1e51fa8640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e300683b0 0x7f1e3006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.149+0000 7f1e51fa8640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e300683b0 0x7f1e3006a870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1e480af000 tx=0x7f1e480a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:38.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.153+0000 7f1e427fc640 1 -- 172.21.15.12:0/1595716640 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e4401f020 con 0x7f1e4c1457a0 2024-04-27T07:04:38.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.309+0000 7f1e5420c640 1 -- 172.21.15.12:0/1595716640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e4c0b1cc0 con 0x7f1e4c1457a0 2024-04-27T07:04:38.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e427fc640 1 -- 172.21.15.12:0/1595716640 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f1e44056e40 con 0x7f1e4c1457a0 2024-04-27T07:04:38.318 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:38.318 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:38.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 -- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e300683b0 msgr2=0x7f1e3006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e300683b0 0x7f1e3006a870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1e480af000 tx=0x7f1e480a7070 comp rx=0 tx=0).stop 2024-04-27T07:04:38.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 -- 172.21.15.12:0/1595716640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c1457a0 msgr2=0x7f1e4c0a9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c1457a0 0x7f1e4c0a9f40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400b440 tx=0x7f1e4400b900 comp rx=0 tx=0).stop 2024-04-27T07:04:38.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 -- 172.21.15.12:0/1595716640 shutdown_connections 2024-04-27T07:04:38.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e300683b0 0x7f1e3006a870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e4c1457a0 0x7f1e4c0a9f40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e4c0b12c0 0x7f1e4c0a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 --2- 172.21.15.12:0/1595716640 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e4c067b70 0x7f1e4c141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 -- 172.21.15.12:0/1595716640 >> 172.21.15.12:0/1595716640 conn(0x7f1e4c0b68c0 msgr2=0x7f1e4c0b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:38.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.313+0000 7f1e5420c640 1 -- 172.21.15.12:0/1595716640 shutdown_connections 2024-04-27T07:04:38.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.317+0000 7f1e5420c640 1 -- 172.21.15.12:0/1595716640 wait complete. 2024-04-27T07:04:38.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:12.742740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:13.912898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:38.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:38.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffbccc640 1 Processor -- start 2024-04-27T07:04:38.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffbccc640 1 -- start start 2024-04-27T07:04:38.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffbccc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9ff40b0ab0 0x7f9ff40733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffbccc640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff4067f50 con 0x7f9ff40729a0 2024-04-27T07:04:38.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffbccc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff40b1350 con 0x7f9ff4067b70 2024-04-27T07:04:38.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffbccc640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff4071ce0 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ff9267640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9ff40729a0 0x7f9ff4144ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43092/0 (socket says 172.21.15.12:43092) 2024-04-27T07:04:38.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ff9267640 1 -- 172.21.15.12:0/2158968648 learned_addr learned my addr 172.21.15.12:0/2158968648 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:38.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffa269640 1 --2- 172.21.15.12:0/2158968648 >> v2:172.21.15.12:3300/0 conn(0x7f9ff40b0ab0 0x7f9ff40733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ff8a66640 1 -- 172.21.15.12:0/2158968648 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1901783600 0 0) 0x7f9ff4067f50 con 0x7f9ff40729a0 2024-04-27T07:04:38.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffa269640 1 -- 172.21.15.12:0/2158968648 >> v1:172.21.15.53:6789/0 conn(0x7f9ff4067b70 legacy=0x7f9ff4147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffa269640 1 -- 172.21.15.12:0/2158968648 >> v1:172.21.15.12:6789/0 conn(0x7f9ff40729a0 legacy=0x7f9ff4144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffa269640 1 -- 172.21.15.12:0/2158968648 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff4071f30 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.453+0000 7f9ffa269640 1 --2- 172.21.15.12:0/2158968648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 0x7f9ff40733b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4071e60 tx=0x7f9fec01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4dd412ea052fd5d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:38.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ff8a66640 1 -- 172.21.15.12:0/2158968648 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0b2070 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ff8a66640 1 -- 172.21.15.12:0/2158968648 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec0a5790 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ff8a66640 1 -- 172.21.15.12:0/2158968648 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0a5b60 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- 172.21.15.12:0/2158968648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 msgr2=0x7f9ff40733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 --2- 172.21.15.12:0/2158968648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 0x7f9ff40733b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4071e60 tx=0x7f9fec01c8a0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- 172.21.15.12:0/2158968648 shutdown_connections 2024-04-27T07:04:38.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 --2- 172.21.15.12:0/2158968648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 0x7f9ff40733b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- 172.21.15.12:0/2158968648 >> 172.21.15.12:0/2158968648 conn(0x7f9ff40b9e80 msgr2=0x7f9ff40ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:38.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- 172.21.15.12:0/2158968648 shutdown_connections 2024-04-27T07:04:38.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- 172.21.15.12:0/2158968648 wait complete. 2024-04-27T07:04:38.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 Processor -- start 2024-04-27T07:04:38.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- start start 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ff4067b70 0x7f9ff40b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9ff40729a0 0x7f9ff41e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 0x7f9ff41e0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff406e190 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff406e010 con 0x7f9ff4067b70 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff406e310 con 0x7f9ff40729a0 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffa269640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 0x7f9ff41e0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffa269640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 0x7f9ff41e0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54718/0 (socket says 172.21.15.12:54718) 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffa269640 1 -- 172.21.15.12:0/3322018585 learned_addr learned my addr 172.21.15.12:0/3322018585 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ff9267640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9ff40729a0 0x7f9ff41e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ff9a68640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ff4067b70 0x7f9ff40b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffa269640 1 -- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9ff40729a0 msgr2=0x7f9ff41e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffa269640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9ff40729a0 0x7f9ff41e08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffa269640 1 -- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ff4067b70 msgr2=0x7f9ff40b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffa269640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ff4067b70 0x7f9ff40b89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffa269640 1 -- 172.21.15.12:0/3322018585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff41e03c0 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffa269640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 0x7f9ff41e0df0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0b0040 tx=0x7f9fec019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:38.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9fea7fc640 1 -- 172.21.15.12:0/3322018585 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0bd020 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9fea7fc640 1 -- 172.21.15.12:0/3322018585 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec0b3040 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- 172.21.15.12:0/3322018585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff40b9350 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9fea7fc640 1 -- 172.21.15.12:0/3322018585 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0b2050 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.457+0000 7f9ffbccc640 1 -- 172.21.15.12:0/3322018585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff40b7ba0 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.461+0000 7f9ffbccc640 1 -- 172.21.15.12:0/3322018585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff41e0590 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.461+0000 7f9fea7fc640 1 -- 172.21.15.12:0/3322018585 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9fec0039a0 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.461+0000 7f9fea7fc640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9fd8068380 0x7f9fd806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.461+0000 7f9fea7fc640 1 -- 172.21.15.12:0/3322018585 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9fec1217f0 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.461+0000 7f9ff9a68640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9fd8068380 0x7f9fd806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.461+0000 7f9ff9a68640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9fd8068380 0x7f9fd806a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0007fd0 tx=0x7f9ff0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:38.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.465+0000 7f9fea7fc640 1 -- 172.21.15.12:0/3322018585 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ff41e0590 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.629+0000 7f9ffbccc640 1 -- 172.21.15.12:0/3322018585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ff40b3370 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9fea7fc640 1 -- 172.21.15.12:0/3322018585 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f9fec0c7060 con 0x7f9ff40b0ab0 2024-04-27T07:04:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:38.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 -- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9fd8068380 msgr2=0x7f9fd806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9fd8068380 0x7f9fd806a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0007fd0 tx=0x7f9ff0009040 comp rx=0 tx=0).stop 2024-04-27T07:04:38.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 -- 172.21.15.12:0/3322018585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 msgr2=0x7f9ff41e0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 0x7f9ff41e0df0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0b0040 tx=0x7f9fec019640 comp rx=0 tx=0).stop 2024-04-27T07:04:38.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 -- 172.21.15.12:0/3322018585 shutdown_connections 2024-04-27T07:04:38.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9fd8068380 0x7f9fd806a840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff40b0ab0 0x7f9ff41e0df0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9ff40729a0 0x7f9ff41e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 --2- 172.21.15.12:0/3322018585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ff4067b70 0x7f9ff40b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 -- 172.21.15.12:0/3322018585 >> 172.21.15.12:0/3322018585 conn(0x7f9ff40b9e80 msgr2=0x7f9ff41443a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:38.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 -- 172.21.15.12:0/3322018585 shutdown_connections 2024-04-27T07:04:38.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.633+0000 7f9ffbccc640 1 -- 172.21.15.12:0/3322018585 wait complete. 2024-04-27T07:04:38.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:04:38.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387eebd640 1 Processor -- start 2024-04-27T07:04:38.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387eebd640 1 -- start start 2024-04-27T07:04:38.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387eebd640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3878141670 0x7f3878141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387eebd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38780b6d80 con 0x7f38780b69a0 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387eebd640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f387809c460 con 0x7f387809d440 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387eebd640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f38780b58d0 con 0x7f3878141670 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387cc59640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f38780b69a0 0x7f3878144330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43108/0 (socket says 172.21.15.12:43108) 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387cc59640 1 -- 172.21.15.12:0/1146929888 learned_addr learned my addr 172.21.15.12:0/1146929888 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f3877fff640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3878141670 0x7f3878141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1603971815 0 0) 0x7f38780b6d80 con 0x7f38780b69a0 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3860003450 con 0x7f38780b69a0 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3852129325 0 0) 0x7f3860003450 con 0x7f38780b69a0 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3878141670 msgr2=0x7f3878141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3878141670 0x7f3878141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 >> v1:172.21.15.53:6789/0 conn(0x7f387809d440 legacy=0x7f387809d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3878144b00 con 0x7f38780b69a0 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f3868001a00 con 0x7f38780b69a0 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 >> v1:172.21.15.12:6789/0 conn(0x7f38780b69a0 legacy=0x7f3878144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3860003ae0 0x7f3860003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3878141670 0x7f386000a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f386000aff0 0x7f386000d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f38780b58d0 con 0x7f386000aff0 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f386000dd00 con 0x7f3860003ae0 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f386000df00 con 0x7f3878141670 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387d45a640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f386000aff0 0x7f386000d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387cc59640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3860003ae0 0x7f3860003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f3877fff640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3878141670 0x7f386000a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387d45a640 1 -- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3878141670 msgr2=0x7f386000a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387d45a640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3878141670 0x7f386000a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387d45a640 1 -- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3860003ae0 msgr2=0x7f3860003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:38.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387d45a640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3860003ae0 0x7f3860003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387d45a640 1 -- 172.21.15.12:0/1146929888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3860003450 con 0x7f386000aff0 2024-04-27T07:04:38.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.781+0000 7f387cc59640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3860003ae0 0x7f3860003f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387d45a640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f386000aff0 0x7f386000d480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3870001750 tx=0x7f387001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d771fa399d3cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38700013a0 con 0x7f386000aff0 2024-04-27T07:04:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38700052f0 con 0x7f386000aff0 2024-04-27T07:04:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f38777fe640 1 -- 172.21.15.12:0/1146929888 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870003840 con 0x7f386000aff0 2024-04-27T07:04:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 -- 172.21.15.12:0/1146929888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f386000aff0 msgr2=0x7f386000d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 --2- 172.21.15.12:0/1146929888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f386000aff0 0x7f386000d480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3870001750 tx=0x7f387001c3e0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387d45a640 1 -- 172.21.15.12:0/1146929888 reap_dead start 2024-04-27T07:04:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 -- 172.21.15.12:0/1146929888 shutdown_connections 2024-04-27T07:04:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 -- 172.21.15.12:0/1146929888 >> 172.21.15.12:0/1146929888 conn(0x7f38780b93b0 msgr2=0x7f38780b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:38.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 -- 172.21.15.12:0/1146929888 shutdown_connections 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 -- 172.21.15.12:0/1146929888 wait complete. 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 Processor -- start 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 -- start start 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38780b2e20 0x7f38780b84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38780b07b0 0x7f38780b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f387813c500 0x7f387813c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f38780b9ea0 con 0x7f387813c500 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f38781450c0 con 0x7f38780b2e20 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387eebd640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f38780ba020 con 0x7f38780b07b0 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387d45a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f387813c500 0x7f387813c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387d45a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f387813c500 0x7f387813c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54738/0 (socket says 172.21.15.12:54738) 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387d45a640 1 -- 172.21.15.12:0/4249610004 learned_addr learned my addr 172.21.15.12:0/4249610004 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387cc59640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38780b2e20 0x7f38780b84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f3877fff640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38780b07b0 0x7f38780b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387d45a640 1 -- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38780b07b0 msgr2=0x7f38780b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387d45a640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38780b07b0 0x7f38780b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387d45a640 1 -- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38780b2e20 msgr2=0x7f38780b84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387d45a640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38780b2e20 0x7f38780b84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.785+0000 7f387d45a640 1 -- 172.21.15.12:0/4249610004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780b2430 con 0x7f387813c500 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f3877fff640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38780b07b0 0x7f38780b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f387d45a640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f387813c500 0x7f387813c980 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f387001cc80 tx=0x7f3860003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f38757fa640 1 -- 172.21.15.12:0/4249610004 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38700013a0 con 0x7f387813c500 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f38757fa640 1 -- 172.21.15.12:0/4249610004 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38700ba070 con 0x7f387813c500 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f387eebd640 1 -- 172.21.15.12:0/4249610004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38780b3200 con 0x7f387813c500 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f38757fa640 1 -- 172.21.15.12:0/4249610004 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870007a40 con 0x7f387813c500 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f387eebd640 1 -- 172.21.15.12:0/4249610004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38780b1040 con 0x7f387813c500 2024-04-27T07:04:38.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f387eebd640 1 -- 172.21.15.12:0/4249610004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3878068c20 con 0x7f387813c500 2024-04-27T07:04:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f38757fa640 1 -- 172.21.15.12:0/4249610004 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f38700052f0 con 0x7f387813c500 2024-04-27T07:04:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f38757fa640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3864068380 0x7f386406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f38757fa640 1 -- 172.21.15.12:0/4249610004 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3870120650 con 0x7f387813c500 2024-04-27T07:04:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f387cc59640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3864068380 0x7f386406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:38.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.789+0000 7f387cc59640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3864068380 0x7f386406a840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3868001590 tx=0x7f38680018a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:38.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.793+0000 7f38757fa640 1 -- 172.21.15.12:0/4249610004 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f38700160f0 con 0x7f387813c500 2024-04-27T07:04:38.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 -- 172.21.15.12:0/4249610004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f387813fcb0 con 0x7f387813c500 2024-04-27T07:04:38.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f38757fa640 1 -- 172.21.15.12:0/4249610004 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f38700f0ca0 con 0x7f387813c500 2024-04-27T07:04:38.957 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:04:38.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 -- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3864068380 msgr2=0x7f386406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3864068380 0x7f386406a840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3868001590 tx=0x7f38680018a0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 -- 172.21.15.12:0/4249610004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f387813c500 msgr2=0x7f387813c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:38.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f387813c500 0x7f387813c980 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f387001cc80 tx=0x7f3860003740 comp rx=0 tx=0).stop 2024-04-27T07:04:38.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 -- 172.21.15.12:0/4249610004 shutdown_connections 2024-04-27T07:04:38.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3864068380 0x7f386406a840 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f387813c500 0x7f387813c980 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38780b07b0 0x7f38780b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 --2- 172.21.15.12:0/4249610004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38780b2e20 0x7f38780b84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:38.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 -- 172.21.15.12:0/4249610004 >> 172.21.15.12:0/4249610004 conn(0x7f38780b93b0 msgr2=0x7f3878140b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:38.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 -- 172.21.15.12:0/4249610004 shutdown_connections 2024-04-27T07:04:38.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:38.953+0000 7f387eebd640 1 -- 172.21.15.12:0/4249610004 wait complete. 2024-04-27T07:04:38.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:04:39.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfefb07640 1 Processor -- start 2024-04-27T07:04:39.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfefb07640 1 -- start start 2024-04-27T07:04:39.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfefb07640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcfe80b5390 0x7fcfe8148600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfefb07640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 0x7fcfe80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfefb07640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfe80b5770 con 0x7fcfe80b6730 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfefb07640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fcfe8147e00 con 0x7fcfe80b5390 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfefb07640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe81480e0 con 0x7fcfe80b8090 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfee0a4640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcfe80b6730 0x7fcfe8141b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43114/0 (socket says 172.21.15.12:43114) 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfee0a4640 1 -- 172.21.15.12:0/1975222200 learned_addr learned my addr 172.21.15.12:0/1975222200 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfed0a2640 1 --2- 172.21.15.12:0/1975222200 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 0x7fcfe80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfed8a3640 1 --2- 172.21.15.12:0/1975222200 >> v2:172.21.15.53:3300/0 conn(0x7fcfe80b5390 0x7fcfe8148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfec8a1640 1 -- 172.21.15.12:0/1975222200 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1109551905 0 0) 0x7fcfe80b5770 con 0x7fcfe80b6730 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfec8a1640 1 -- 172.21.15.12:0/1975222200 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfd4003530 con 0x7fcfe80b6730 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfec8a1640 1 -- 172.21.15.12:0/1975222200 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1915200795 0 0) 0x7fcfd4003530 con 0x7fcfe80b6730 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfed0a2640 1 -- 172.21.15.12:0/1975222200 >> v2:172.21.15.53:3300/0 conn(0x7fcfe80b5390 msgr2=0x7fcfe8148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfed0a2640 1 --2- 172.21.15.12:0/1975222200 >> v2:172.21.15.53:3300/0 conn(0x7fcfe80b5390 0x7fcfe8148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfed0a2640 1 -- 172.21.15.12:0/1975222200 >> v1:172.21.15.12:6789/0 conn(0x7fcfe80b6730 legacy=0x7fcfe8141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.097+0000 7fcfed0a2640 1 -- 172.21.15.12:0/1975222200 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe8148b40 con 0x7fcfe80b8090 2024-04-27T07:04:39.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfed0a2640 1 --2- 172.21.15.12:0/1975222200 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 0x7fcfe80b61f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80097c0 tx=0x7fcfd801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4a152f8241999bf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:39.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfec8a1640 1 -- 172.21.15.12:0/1975222200 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80b0070 con 0x7fcfe80b8090 2024-04-27T07:04:39.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfec8a1640 1 -- 172.21.15.12:0/1975222200 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfd80ab400 con 0x7fcfe80b8090 2024-04-27T07:04:39.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfec8a1640 1 -- 172.21.15.12:0/1975222200 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4001680 con 0x7fcfe80b8090 2024-04-27T07:04:39.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 -- 172.21.15.12:0/1975222200 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 msgr2=0x7fcfe80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 --2- 172.21.15.12:0/1975222200 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 0x7fcfe80b61f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80097c0 tx=0x7fcfd801c330 comp rx=0 tx=0).stop 2024-04-27T07:04:39.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 -- 172.21.15.12:0/1975222200 shutdown_connections 2024-04-27T07:04:39.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 --2- 172.21.15.12:0/1975222200 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 0x7fcfe80b61f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 --2- 172.21.15.12:0/1975222200 >> v2:172.21.15.53:3300/0 conn(0x7fcfe80b5390 0x7fcfe8148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 -- 172.21.15.12:0/1975222200 >> 172.21.15.12:0/1975222200 conn(0x7fcfe80bd690 msgr2=0x7fcfe80bdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:39.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 -- 172.21.15.12:0/1975222200 shutdown_connections 2024-04-27T07:04:39.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 -- 172.21.15.12:0/1975222200 wait complete. 2024-04-27T07:04:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 Processor -- start 2024-04-27T07:04:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 -- start start 2024-04-27T07:04:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe80b5390 0x7fcfe80bc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 0x7fcfe81e0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfe8143520 0x7fcfe81e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8155120 con 0x7fcfe8143520 2024-04-27T07:04:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8154fa0 con 0x7fcfe80b5390 2024-04-27T07:04:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfefb07640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fcfe81552a0 con 0x7fcfe80b8090 2024-04-27T07:04:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfee0a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfe8143520 0x7fcfe81e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfee0a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfe8143520 0x7fcfe81e0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54752/0 (socket says 172.21.15.12:54752) 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfee0a4640 1 -- 172.21.15.12:0/2800641726 learned_addr learned my addr 172.21.15.12:0/2800641726 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.101+0000 7fcfed8a3640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe80b5390 0x7fcfe80bc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfed0a2640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 0x7fcfe81e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfee0a4640 1 -- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 msgr2=0x7fcfe81e0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfee0a4640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 0x7fcfe81e0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfee0a4640 1 -- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe80b5390 msgr2=0x7fcfe80bc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfee0a4640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe80b5390 0x7fcfe80bc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfee0a4640 1 -- 172.21.15.12:0/2800641726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe8143a50 con 0x7fcfe8143520 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfee0a4640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfe8143520 0x7fcfe81e0ba0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40011f0 tx=0x7fcfe400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfde7fc640 1 -- 172.21.15.12:0/2800641726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4005ae0 con 0x7fcfe8143520 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfde7fc640 1 -- 172.21.15.12:0/2800641726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe4005ca0 con 0x7fcfe8143520 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfde7fc640 1 -- 172.21.15.12:0/2800641726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe40072b0 con 0x7fcfe8143520 2024-04-27T07:04:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfefb07640 1 -- 172.21.15.12:0/2800641726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe813a760 con 0x7fcfe8143520 2024-04-27T07:04:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfefb07640 1 -- 172.21.15.12:0/2800641726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe80bd110 con 0x7fcfe8143520 2024-04-27T07:04:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfefb07640 1 -- 172.21.15.12:0/2800641726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe8069330 con 0x7fcfe8143520 2024-04-27T07:04:39.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfde7fc640 1 -- 172.21.15.12:0/2800641726 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcfe4007450 con 0x7fcfe8143520 2024-04-27T07:04:39.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfde7fc640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcfcc068380 0x7fcfcc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfde7fc640 1 -- 172.21.15.12:0/2800641726 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcfe40867f0 con 0x7fcfe8143520 2024-04-27T07:04:39.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfed8a3640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcfcc068380 0x7fcfcc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.105+0000 7fcfed8a3640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcfcc068380 0x7fcfcc06a840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00074a0 tx=0x7fcfe000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.109+0000 7fcfde7fc640 1 -- 172.21.15.12:0/2800641726 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcfe4056da0 con 0x7fcfe8143520 2024-04-27T07:04:39.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 -- 172.21.15.12:0/2800641726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcfe81429d0 con 0x7fcfe8143520 2024-04-27T07:04:39.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfde7fc640 1 -- 172.21.15.12:0/2800641726 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fcfe40566a0 con 0x7fcfe8143520 2024-04-27T07:04:39.278 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:04:39.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 -- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcfcc068380 msgr2=0x7fcfcc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcfcc068380 0x7fcfcc06a840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00074a0 tx=0x7fcfe000b040 comp rx=0 tx=0).stop 2024-04-27T07:04:39.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 -- 172.21.15.12:0/2800641726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfe8143520 msgr2=0x7fcfe81e0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfe8143520 0x7fcfe81e0ba0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40011f0 tx=0x7fcfe400cbe0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 -- 172.21.15.12:0/2800641726 shutdown_connections 2024-04-27T07:04:39.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcfcc068380 0x7fcfcc06a840 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfe8143520 0x7fcfe81e0ba0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcfe80b8090 0x7fcfe81e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 --2- 172.21.15.12:0/2800641726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfe80b5390 0x7fcfe80bc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 -- 172.21.15.12:0/2800641726 >> 172.21.15.12:0/2800641726 conn(0x7fcfe80bd690 msgr2=0x7fcfe813b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:39.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 -- 172.21.15.12:0/2800641726 shutdown_connections 2024-04-27T07:04:39.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.273+0000 7fcfefb07640 1 -- 172.21.15.12:0/2800641726 wait complete. 2024-04-27T07:04:39.298 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:39.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.409+0000 7f521bd35640 1 Processor -- start 2024-04-27T07:04:39.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.409+0000 7f521bd35640 1 -- start start 2024-04-27T07:04:39.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521bd35640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52140b64c0 0x7f52140b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521bd35640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214152640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521bd35640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52140b7560 con 0x7f52140b5ae0 2024-04-27T07:04:39.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521bd35640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5214148ca0 con 0x7f52140b64c0 2024-04-27T07:04:39.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521bd35640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f52140b77d0 con 0x7f5214148810 2024-04-27T07:04:39.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521a2d2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f5219ad1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f52140b64c0 0x7f52140b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521a2d2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:32862/0 (socket says 172.21.15.12:32862) 2024-04-27T07:04:39.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521a2d2640 1 -- 172.21.15.12:0/3202014634 learned_addr learned my addr 172.21.15.12:0/3202014634 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:39.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521a2d2640 1 -- 172.21.15.12:0/3202014634 >> v2:172.21.15.53:3300/0 conn(0x7f52140b64c0 msgr2=0x7f52140b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:39.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521a2d2640 1 --2- 172.21.15.12:0/3202014634 >> v2:172.21.15.53:3300/0 conn(0x7f52140b64c0 0x7f52140b7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521a2d2640 1 -- 172.21.15.12:0/3202014634 >> v1:172.21.15.53:6789/0 conn(0x7f52140b5ae0 legacy=0x7f5214141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521a2d2640 1 -- 172.21.15.12:0/3202014634 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52140b68a0 con 0x7f5214148810 2024-04-27T07:04:39.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.413+0000 7f521a2d2640 1 --2- 172.21.15.12:0/3202014634 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214152640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f520c009e50 tx=0x7f520c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fdbd9a1b5eb3129 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:39.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f5218acf640 1 -- 172.21.15.12:0/3202014634 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c0a56b0 con 0x7f5214148810 2024-04-27T07:04:39.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f5218acf640 1 -- 172.21.15.12:0/3202014634 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520c0a5850 con 0x7f5214148810 2024-04-27T07:04:39.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f5218acf640 1 -- 172.21.15.12:0/3202014634 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c0a5c20 con 0x7f5214148810 2024-04-27T07:04:39.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 -- 172.21.15.12:0/3202014634 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 msgr2=0x7f5214152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 --2- 172.21.15.12:0/3202014634 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214152640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f520c009e50 tx=0x7f520c01c8e0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 -- 172.21.15.12:0/3202014634 shutdown_connections 2024-04-27T07:04:39.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 --2- 172.21.15.12:0/3202014634 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214152640 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 --2- 172.21.15.12:0/3202014634 >> v2:172.21.15.53:3300/0 conn(0x7f52140b64c0 0x7f52140b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 -- 172.21.15.12:0/3202014634 >> 172.21.15.12:0/3202014634 conn(0x7f5214073c00 msgr2=0x7f52140bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:39.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 -- 172.21.15.12:0/3202014634 shutdown_connections 2024-04-27T07:04:39.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 -- 172.21.15.12:0/3202014634 wait complete. 2024-04-27T07:04:39.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 Processor -- start 2024-04-27T07:04:39.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 -- start start 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140b5ae0 0x7f52141453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52140b64c0 0x7f52141e0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214137700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f52140b51c0 con 0x7f52140b64c0 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52140b5040 con 0x7f52140b5ae0 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521bd35640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f52141421e0 con 0x7f5214148810 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f52192d0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52140b64c0 0x7f52141e0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f52192d0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52140b64c0 0x7f52141e0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54754/0 (socket says 172.21.15.12:54754) 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f52192d0640 1 -- 172.21.15.12:0/1948936214 learned_addr learned my addr 172.21.15.12:0/1948936214 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521a2d2640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214137700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f5219ad1640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140b5ae0 0x7f52141453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f52192d0640 1 -- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 msgr2=0x7f5214137700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f52192d0640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214137700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f52192d0640 1 -- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140b5ae0 msgr2=0x7f52141453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f52192d0640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140b5ae0 0x7f52141453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f52192d0640 1 -- 172.21.15.12:0/1948936214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52141e3c80 con 0x7f52140b64c0 2024-04-27T07:04:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.417+0000 7f521a2d2640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214137700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f52192d0640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52140b64c0 0x7f52141e0570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f520800c370 tx=0x7f520800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f52067fc640 1 -- 172.21.15.12:0/1948936214 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52080052d0 con 0x7f52140b64c0 2024-04-27T07:04:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f52067fc640 1 -- 172.21.15.12:0/1948936214 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5208005470 con 0x7f52140b64c0 2024-04-27T07:04:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f52067fc640 1 -- 172.21.15.12:0/1948936214 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208007260 con 0x7f52140b64c0 2024-04-27T07:04:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f521bd35640 1 -- 172.21.15.12:0/1948936214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52141e3f60 con 0x7f52140b64c0 2024-04-27T07:04:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f521bd35640 1 -- 172.21.15.12:0/1948936214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5214144960 con 0x7f52140b64c0 2024-04-27T07:04:39.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f521bd35640 1 -- 172.21.15.12:0/1948936214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5214068c20 con 0x7f52140b64c0 2024-04-27T07:04:39.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f52067fc640 1 -- 172.21.15.12:0/1948936214 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5208007400 con 0x7f52140b64c0 2024-04-27T07:04:39.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f52067fc640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f51f8068380 0x7f51f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f52067fc640 1 -- 172.21.15.12:0/1948936214 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5208085db0 con 0x7f52140b64c0 2024-04-27T07:04:39.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f5219ad1640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f51f8068380 0x7f51f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.421+0000 7f5219ad1640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f51f8068380 0x7f51f806a840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5210009a50 tx=0x7f5210008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:39.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.429+0000 7f52067fc640 1 -- 172.21.15.12:0/1948936214 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5208056400 con 0x7f52140b64c0 2024-04-27T07:04:39.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.593+0000 7f521bd35640 1 -- 172.21.15.12:0/1948936214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52140b53e0 con 0x7f52140b64c0 2024-04-27T07:04:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.593+0000 7f52067fc640 1 -- 172.21.15.12:0/1948936214 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f5208055d00 con 0x7f52140b64c0 2024-04-27T07:04:39.598 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:39.598 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:39.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.593+0000 7f521bd35640 1 -- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f51f8068380 msgr2=0x7f51f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.593+0000 7f521bd35640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f51f8068380 0x7f51f806a840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5210009a50 tx=0x7f5210008040 comp rx=0 tx=0).stop 2024-04-27T07:04:39.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.593+0000 7f521bd35640 1 -- 172.21.15.12:0/1948936214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52140b64c0 msgr2=0x7f52141e0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.593+0000 7f521bd35640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52140b64c0 0x7f52141e0570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f520800c370 tx=0x7f520800c830 comp rx=0 tx=0).stop 2024-04-27T07:04:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.597+0000 7f521bd35640 1 -- 172.21.15.12:0/1948936214 shutdown_connections 2024-04-27T07:04:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.597+0000 7f521bd35640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f51f8068380 0x7f51f806a840 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.597+0000 7f521bd35640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5214148810 0x7f5214137700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.597+0000 7f521bd35640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52140b64c0 0x7f52141e0570 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.597+0000 7f521bd35640 1 --2- 172.21.15.12:0/1948936214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52140b5ae0 0x7f52141453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.597+0000 7f521bd35640 1 -- 172.21.15.12:0/1948936214 >> 172.21.15.12:0/1948936214 conn(0x7f5214073c00 msgr2=0x7f52140bd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.597+0000 7f521bd35640 1 -- 172.21.15.12:0/1948936214 shutdown_connections 2024-04-27T07:04:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.597+0000 7f521bd35640 1 -- 172.21.15.12:0/1948936214 wait complete. 2024-04-27T07:04:39.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:12.742740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:13.912898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:39.622 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:39.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b4cb57640 1 Processor -- start 2024-04-27T07:04:39.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b4cb57640 1 -- start start 2024-04-27T07:04:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b4cb57640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0b480b8cc0 0x7f0b48144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b4cb57640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0b481453a0 0x7f0b48145830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b4cb57640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b480ba470 con 0x7f0b480b73c0 2024-04-27T07:04:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b4cb57640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f0b480b77a0 con 0x7f0b480b8cc0 2024-04-27T07:04:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b4cb57640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0b480b5c90 con 0x7f0b481453a0 2024-04-27T07:04:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b45d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0b480b8cc0 0x7f0b48144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b45d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0b480b8cc0 0x7f0b48144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54758/0 (socket says 172.21.15.12:54758) 2024-04-27T07:04:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b45d9b640 1 -- 172.21.15.12:0/3044638329 learned_addr learned my addr 172.21.15.12:0/3044638329 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b46d9d640 1 --2- 172.21.15.12:0/3044638329 >> v2:172.21.15.53:3300/0 conn(0x7f0b481453a0 0x7f0b48145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b45d9b640 1 -- 172.21.15.12:0/3044638329 >> v2:172.21.15.53:3300/0 conn(0x7f0b481453a0 msgr2=0x7f0b48145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b45d9b640 1 --2- 172.21.15.12:0/3044638329 >> v2:172.21.15.53:3300/0 conn(0x7f0b481453a0 0x7f0b48145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b45d9b640 1 -- 172.21.15.12:0/3044638329 >> v1:172.21.15.53:6789/0 conn(0x7f0b480b73c0 legacy=0x7f0b480b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:39.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.749+0000 7f0b45d9b640 1 -- 172.21.15.12:0/3044638329 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b480b5f70 con 0x7f0b480b8cc0 2024-04-27T07:04:39.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b45d9b640 1 --2- 172.21.15.12:0/3044638329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 0x7f0b48144e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b480b6640 tx=0x7f0b3c01c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfeb7694f7c0d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:39.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4559a640 1 -- 172.21.15.12:0/3044638329 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0b0070 con 0x7f0b480b8cc0 2024-04-27T07:04:39.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4559a640 1 -- 172.21.15.12:0/3044638329 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0ab600 con 0x7f0b480b8cc0 2024-04-27T07:04:39.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4559a640 1 -- 172.21.15.12:0/3044638329 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0ab9d0 con 0x7f0b480b8cc0 2024-04-27T07:04:39.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 -- 172.21.15.12:0/3044638329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 msgr2=0x7f0b48144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 --2- 172.21.15.12:0/3044638329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 0x7f0b48144e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b480b6640 tx=0x7f0b3c01c1e0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 -- 172.21.15.12:0/3044638329 shutdown_connections 2024-04-27T07:04:39.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 --2- 172.21.15.12:0/3044638329 >> v2:172.21.15.53:3300/0 conn(0x7f0b481453a0 0x7f0b48145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 --2- 172.21.15.12:0/3044638329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 0x7f0b48144e60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 -- 172.21.15.12:0/3044638329 >> 172.21.15.12:0/3044638329 conn(0x7f0b480ba6d0 msgr2=0x7f0b480baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:39.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 -- 172.21.15.12:0/3044638329 shutdown_connections 2024-04-27T07:04:39.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 -- 172.21.15.12:0/3044638329 wait complete. 2024-04-27T07:04:39.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 Processor -- start 2024-04-27T07:04:39.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 -- start start 2024-04-27T07:04:39.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0b480b73c0 0x7f0b4809ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 0x7f0b4809dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b481453a0 0x7f0b480afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4809fc30 con 0x7f0b480b8cc0 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4809fab0 con 0x7f0b481453a0 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4cb57640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4814ff60 con 0x7f0b480b73c0 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b45d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 0x7f0b4809dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b45d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 0x7f0b4809dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54768/0 (socket says 172.21.15.12:54768) 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b45d9b640 1 -- 172.21.15.12:0/466675574 learned_addr learned my addr 172.21.15.12:0/466675574 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b46d9d640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b481453a0 0x7f0b480afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4659c640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0b480b73c0 0x7f0b4809ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b45d9b640 1 -- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0b480b73c0 msgr2=0x7f0b4809ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b45d9b640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0b480b73c0 0x7f0b4809ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b45d9b640 1 -- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b481453a0 msgr2=0x7f0b480afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b45d9b640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b481453a0 0x7f0b480afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b45d9b640 1 -- 172.21.15.12:0/466675574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b48140df0 con 0x7f0b480b8cc0 2024-04-27T07:04:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b46d9d640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b481453a0 0x7f0b480afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b4659c640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0b480b73c0 0x7f0b4809ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b45d9b640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 0x7f0b4809dab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c01a890 tx=0x7f0b3c01aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b36ffd640 1 -- 172.21.15.12:0/466675574 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0ba020 con 0x7f0b480b8cc0 2024-04-27T07:04:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b36ffd640 1 -- 172.21.15.12:0/466675574 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0b6640 con 0x7f0b480b8cc0 2024-04-27T07:04:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.753+0000 7f0b36ffd640 1 -- 172.21.15.12:0/466675574 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0b0040 con 0x7f0b480b8cc0 2024-04-27T07:04:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.757+0000 7f0b4cb57640 1 -- 172.21.15.12:0/466675574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b480b0690 con 0x7f0b480b8cc0 2024-04-27T07:04:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.757+0000 7f0b4cb57640 1 -- 172.21.15.12:0/466675574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b480b4b00 con 0x7f0b480b8cc0 2024-04-27T07:04:39.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.757+0000 7f0b4cb57640 1 -- 172.21.15.12:0/466675574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b48069330 con 0x7f0b480b8cc0 2024-04-27T07:04:39.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.757+0000 7f0b36ffd640 1 -- 172.21.15.12:0/466675574 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0b3c0bd020 con 0x7f0b480b8cc0 2024-04-27T07:04:39.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.757+0000 7f0b36ffd640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0b28068380 0x7f0b2806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:39.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.757+0000 7f0b36ffd640 1 -- 172.21.15.12:0/466675574 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0b3c1208d0 con 0x7f0b480b8cc0 2024-04-27T07:04:39.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.757+0000 7f0b4659c640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0b28068380 0x7f0b2806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:39.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.757+0000 7f0b4659c640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0b28068380 0x7f0b2806a840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0b38003e40 tx=0x7f0b38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:39.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.761+0000 7f0b36ffd640 1 -- 172.21.15.12:0/466675574 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0b3c0c4070 con 0x7f0b480b8cc0 2024-04-27T07:04:39.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.925+0000 7f0b4cb57640 1 -- 172.21.15.12:0/466675574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b48073150 con 0x7f0b480b8cc0 2024-04-27T07:04:39.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.925+0000 7f0b36ffd640 1 -- 172.21.15.12:0/466675574 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0b480ba470 con 0x7f0b480b8cc0 2024-04-27T07:04:39.930 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 -- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0b28068380 msgr2=0x7f0b2806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0b28068380 0x7f0b2806a840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0b38003e40 tx=0x7f0b38009040 comp rx=0 tx=0).stop 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 -- 172.21.15.12:0/466675574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 msgr2=0x7f0b4809dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 0x7f0b4809dab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c01a890 tx=0x7f0b3c01aa20 comp rx=0 tx=0).stop 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 -- 172.21.15.12:0/466675574 shutdown_connections 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0b28068380 0x7f0b2806a840 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b481453a0 0x7f0b480afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b480b8cc0 0x7f0b4809dab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 --2- 172.21.15.12:0/466675574 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0b480b73c0 0x7f0b4809ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 -- 172.21.15.12:0/466675574 >> 172.21.15.12:0/466675574 conn(0x7f0b480ba6d0 msgr2=0x7f0b4806bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 -- 172.21.15.12:0/466675574 shutdown_connections 2024-04-27T07:04:39.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:39.929+0000 7f0b4cb57640 1 -- 172.21.15.12:0/466675574 wait complete. 2024-04-27T07:04:39.951 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:04:40.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41fe2e640 1 Processor -- start 2024-04-27T07:04:40.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41fe2e640 1 -- start start 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41fe2e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb4181489c0 0x7fb4180b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41fe2e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fb418142010 0x7fb4180b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41fe2e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4180b6860 con 0x7fb4180b6480 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41fe2e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fb4180b84e0 con 0x7fb4181489c0 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41fe2e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fb418147ee0 con 0x7fb418142010 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41d3c9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb4181489c0 0x7fb4180b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41dbca640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb4180b6480 0x7fb418141900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43130/0 (socket says 172.21.15.12:43130) 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41dbca640 1 -- 172.21.15.12:0/4079089069 learned_addr learned my addr 172.21.15.12:0/4079089069 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41e3cb640 1 --2- 172.21.15.12:0/4079089069 >> v2:172.21.15.53:3300/0 conn(0x7fb418142010 0x7fb4180b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41d3c9640 1 -- 172.21.15.12:0/4079089069 >> v2:172.21.15.53:3300/0 conn(0x7fb418142010 msgr2=0x7fb4180b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41d3c9640 1 --2- 172.21.15.12:0/4079089069 >> v2:172.21.15.53:3300/0 conn(0x7fb418142010 0x7fb4180b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41d3c9640 1 -- 172.21.15.12:0/4079089069 >> v1:172.21.15.12:6789/0 conn(0x7fb4180b6480 legacy=0x7fb418141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41cbc8640 1 -- 172.21.15.12:0/4079089069 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3235722169 0 0) 0x7fb4180b6860 con 0x7fb4180b6480 2024-04-27T07:04:40.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.077+0000 7fb41d3c9640 1 -- 172.21.15.12:0/4079089069 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4181481c0 con 0x7fb4181489c0 2024-04-27T07:04:40.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41d3c9640 1 --2- 172.21.15.12:0/4079089069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4181489c0 0x7fb4180b5ae0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4180b55b0 tx=0x7fb40801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ff8c6642b7e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:40.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41cbc8640 1 -- 172.21.15.12:0/4079089069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080b0070 con 0x7fb4181489c0 2024-04-27T07:04:40.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41cbc8640 1 -- 172.21.15.12:0/4079089069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4080ab600 con 0x7fb4181489c0 2024-04-27T07:04:40.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41cbc8640 1 -- 172.21.15.12:0/4079089069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080ab9d0 con 0x7fb4181489c0 2024-04-27T07:04:40.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4079089069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4181489c0 msgr2=0x7fb4180b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 --2- 172.21.15.12:0/4079089069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4181489c0 0x7fb4180b5ae0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4180b55b0 tx=0x7fb40801c3f0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4079089069 shutdown_connections 2024-04-27T07:04:40.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 --2- 172.21.15.12:0/4079089069 >> v2:172.21.15.53:3300/0 conn(0x7fb418142010 0x7fb4180b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 --2- 172.21.15.12:0/4079089069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4181489c0 0x7fb4180b5ae0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4079089069 >> 172.21.15.12:0/4079089069 conn(0x7fb4180bd620 msgr2=0x7fb4180bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:40.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4079089069 shutdown_connections 2024-04-27T07:04:40.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4079089069 wait complete. 2024-04-27T07:04:40.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 Processor -- start 2024-04-27T07:04:40.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 -- start start 2024-04-27T07:04:40.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4180b6480 0x7fb41813e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb418142010 0x7fb4181e0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4181489c0 0x7fb4181e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb418154e70 con 0x7fb4180b6480 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb418142420 con 0x7fb4181489c0 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41fe2e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fb418154ff0 con 0x7fb418142010 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41dbca640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4180b6480 0x7fb41813e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41dbca640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4180b6480 0x7fb41813e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54792/0 (socket says 172.21.15.12:54792) 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.081+0000 7fb41dbca640 1 -- 172.21.15.12:0/4133973710 learned_addr learned my addr 172.21.15.12:0/4133973710 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41e3cb640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4181489c0 0x7fb4181e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41d3c9640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb418142010 0x7fb4181e0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41dbca640 1 -- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb418142010 msgr2=0x7fb4181e0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41dbca640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb418142010 0x7fb4181e0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41dbca640 1 -- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4181489c0 msgr2=0x7fb4181e0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41dbca640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4181489c0 0x7fb4181e0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41dbca640 1 -- 172.21.15.12:0/4133973710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb418143c90 con 0x7fb4180b6480 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41e3cb640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4181489c0 0x7fb4181e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:40.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41dbca640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4180b6480 0x7fb41813e0e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c480 tx=0x7fb41400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:40.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb40e7fc640 1 -- 172.21.15.12:0/4133973710 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414018070 con 0x7fb4180b6480 2024-04-27T07:04:40.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb40e7fc640 1 -- 172.21.15.12:0/4133973710 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb414007500 con 0x7fb4180b6480 2024-04-27T07:04:40.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4133973710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb418142810 con 0x7fb4180b6480 2024-04-27T07:04:40.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb40e7fc640 1 -- 172.21.15.12:0/4133973710 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414004a30 con 0x7fb4180b6480 2024-04-27T07:04:40.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4133973710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181439b0 con 0x7fb4180b6480 2024-04-27T07:04:40.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4133973710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb418068c20 con 0x7fb4180b6480 2024-04-27T07:04:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb40e7fc640 1 -- 172.21.15.12:0/4133973710 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb41401b070 con 0x7fb4180b6480 2024-04-27T07:04:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb40e7fc640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3fc068380 0x7fb3fc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb40e7fc640 1 -- 172.21.15.12:0/4133973710 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb414085cc0 con 0x7fb4180b6480 2024-04-27T07:04:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41d3c9640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3fc068380 0x7fb3fc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.085+0000 7fb41d3c9640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3fc068380 0x7fb3fc06a840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb4080ad000 tx=0x7fb4080a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.089+0000 7fb40e7fc640 1 -- 172.21.15.12:0/4133973710 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb414056310 con 0x7fb4180b6480 2024-04-27T07:04:40.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.249+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4133973710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb418140da0 con 0x7fb4180b6480 2024-04-27T07:04:40.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb40e7fc640 1 -- 172.21.15.12:0/4133973710 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb414055c10 con 0x7fb4180b6480 2024-04-27T07:04:40.256 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3fc068380 msgr2=0x7fb3fc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3fc068380 0x7fb3fc06a840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb4080ad000 tx=0x7fb4080a6000 comp rx=0 tx=0).stop 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4133973710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4180b6480 msgr2=0x7fb41813e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4180b6480 0x7fb41813e0e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c480 tx=0x7fb41400c940 comp rx=0 tx=0).stop 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4133973710 shutdown_connections 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb3fc068380 0x7fb3fc06a840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4181489c0 0x7fb4181e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb418142010 0x7fb4181e0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 --2- 172.21.15.12:0/4133973710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4180b6480 0x7fb41813e0e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4133973710 >> 172.21.15.12:0/4133973710 conn(0x7fb4180bd620 msgr2=0x7fb4181416d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4133973710 shutdown_connections 2024-04-27T07:04:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.253+0000 7fb41fe2e640 1 -- 172.21.15.12:0/4133973710 wait complete. 2024-04-27T07:04:40.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:04:40.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.385+0000 7fbf7188a640 1 Processor -- start 2024-04-27T07:04:40.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.385+0000 7fbf7188a640 1 -- start start 2024-04-27T07:04:40.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf7188a640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf6c0b6610 0x7fbf6c1489d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf7188a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6c0b79b0 con 0x7fbf6c1419d0 2024-04-27T07:04:40.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf7188a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6c0b8480 con 0x7fbf6c0b8020 2024-04-27T07:04:40.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf7188a640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c0b69f0 con 0x7fbf6c0b6610 2024-04-27T07:04:40.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf6b7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbf6c1419d0 0x7fbf6c141dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43146/0 (socket says 172.21.15.12:43146) 2024-04-27T07:04:40.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf6b7fe640 1 -- 172.21.15.12:0/1133607903 learned_addr learned my addr 172.21.15.12:0/1133607903 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf6affd640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf6c0b6610 0x7fbf6c1489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1695927680 0 0) 0x7fbf6c0b79b0 con 0x7fbf6c1419d0 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf54003530 con 0x7fbf6c1419d0 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2535351755 0 0) 0x7fbf54003530 con 0x7fbf6c1419d0 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf6c0b6610 msgr2=0x7fbf6c1489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf6c0b6610 0x7fbf6c1489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 >> v1:172.21.15.53:6789/0 conn(0x7fbf6c0b8020 legacy=0x7fbf6c0b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c147ef0 con 0x7fbf6c1419d0 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fbf64001a00 con 0x7fbf6c1419d0 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 >> v1:172.21.15.12:6789/0 conn(0x7fbf6c1419d0 legacy=0x7fbf6c141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf540039f0 0x7fbf54003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf54004810 0x7fbf54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf6c0b6610 0x7fbf54005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0b69f0 con 0x7fbf540039f0 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5400bb00 con 0x7fbf54004810 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5400bd00 con 0x7fbf6c0b6610 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf6affd640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf540039f0 0x7fbf54003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf6a7fc640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf54004810 0x7fbf54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf6affd640 1 -- 172.21.15.12:0/1133607903 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf6c0b6610 msgr2=0x7fbf54005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf6affd640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf6c0b6610 0x7fbf54005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf6affd640 1 -- 172.21.15.12:0/1133607903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf54004810 msgr2=0x7fbf54004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf6affd640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf54004810 0x7fbf54004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.389+0000 7fbf6affd640 1 -- 172.21.15.12:0/1133607903 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf54003530 con 0x7fbf540039f0 2024-04-27T07:04:40.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6affd640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf540039f0 0x7fbf54003e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009ae0 tx=0x7fbf5c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a279d3f74eca7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0096b0 con 0x7fbf540039f0 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c0a6490 con 0x7fbf540039f0 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf69ffb640 1 -- 172.21.15.12:0/1133607903 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0a6880 con 0x7fbf540039f0 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- 172.21.15.12:0/1133607903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf540039f0 msgr2=0x7fbf54003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 --2- 172.21.15.12:0/1133607903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf540039f0 0x7fbf54003e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009ae0 tx=0x7fbf5c01cab0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6b7fe640 1 -- 172.21.15.12:0/1133607903 reap_dead start 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- 172.21.15.12:0/1133607903 shutdown_connections 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- 172.21.15.12:0/1133607903 >> 172.21.15.12:0/1133607903 conn(0x7fbf6c0ba140 msgr2=0x7fbf6c0ba550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- 172.21.15.12:0/1133607903 shutdown_connections 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- 172.21.15.12:0/1133607903 wait complete. 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 Processor -- start 2024-04-27T07:04:40.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- start start 2024-04-27T07:04:40.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf540039f0 0x7fbf6c13ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf6c143240 0x7fbf6c1e0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf6c1e0a40 0x7fbf6c1e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c154070 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c153ef0 con 0x7fbf6c143240 2024-04-27T07:04:40.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c1541f0 con 0x7fbf540039f0 2024-04-27T07:04:40.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf6c1e0a40 0x7fbf6c1e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf6c1e0a40 0x7fbf6c1e3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54812/0 (socket says 172.21.15.12:54812) 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6b7fe640 1 -- 172.21.15.12:0/4222557986 learned_addr learned my addr 172.21.15.12:0/4222557986 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6affd640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf540039f0 0x7fbf6c13ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6a7fc640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf6c143240 0x7fbf6c1e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6b7fe640 1 -- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf540039f0 msgr2=0x7fbf6c13ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6b7fe640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf540039f0 0x7fbf6c13ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6b7fe640 1 -- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf6c143240 msgr2=0x7fbf6c1e0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6b7fe640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf6c143240 0x7fbf6c1e0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6b7fe640 1 -- 172.21.15.12:0/4222557986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c143700 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6a7fc640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf6c143240 0x7fbf6c1e0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf6b7fe640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf6c1e0a40 0x7fbf6c1e3a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0b6d50 tx=0x7fbf6400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf53fff640 1 -- 172.21.15.12:0/4222557986 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400a610 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf53fff640 1 -- 172.21.15.12:0/4222557986 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6400a7b0 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- 172.21.15.12:0/4222557986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c1439e0 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf53fff640 1 -- 172.21.15.12:0/4222557986 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400ab80 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.393+0000 7fbf7188a640 1 -- 172.21.15.12:0/4222557986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c14a7e0 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.397+0000 7fbf7188a640 1 -- 172.21.15.12:0/4222557986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c069330 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.397+0000 7fbf53fff640 1 -- 172.21.15.12:0/4222557986 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbf64011070 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.397+0000 7fbf53fff640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbf58068380 0x7fbf5806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.397+0000 7fbf53fff640 1 -- 172.21.15.12:0/4222557986 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbf64086330 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.397+0000 7fbf6affd640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbf58068380 0x7fbf5806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.397+0000 7fbf6affd640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbf58068380 0x7fbf5806a840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0ad000 tx=0x7fbf5c0b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:40.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.401+0000 7fbf53fff640 1 -- 172.21.15.12:0/4222557986 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbf6401a060 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.557+0000 7fbf7188a640 1 -- 172.21.15.12:0/4222557986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbf6c145770 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf53fff640 1 -- 172.21.15.12:0/4222557986 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fbf64056980 con 0x7fbf6c1e0a40 2024-04-27T07:04:40.564 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:04:40.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 -- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbf58068380 msgr2=0x7fbf5806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbf58068380 0x7fbf5806a840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0ad000 tx=0x7fbf5c0b3040 comp rx=0 tx=0).stop 2024-04-27T07:04:40.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 -- 172.21.15.12:0/4222557986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf6c1e0a40 msgr2=0x7fbf6c1e3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf6c1e0a40 0x7fbf6c1e3a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0b6d50 tx=0x7fbf6400ccd0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 -- 172.21.15.12:0/4222557986 shutdown_connections 2024-04-27T07:04:40.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbf58068380 0x7fbf5806a840 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf6c1e0a40 0x7fbf6c1e3a70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf6c143240 0x7fbf6c1e0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 --2- 172.21.15.12:0/4222557986 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf540039f0 0x7fbf6c13ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 -- 172.21.15.12:0/4222557986 >> 172.21.15.12:0/4222557986 conn(0x7fbf6c0ba140 msgr2=0x7fbf6c1464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:40.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 -- 172.21.15.12:0/4222557986 shutdown_connections 2024-04-27T07:04:40.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.561+0000 7fbf7188a640 1 -- 172.21.15.12:0/4222557986 wait complete. 2024-04-27T07:04:40.583 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:40.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29bd972640 1 Processor -- start 2024-04-27T07:04:40.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29bd972640 1 -- start start 2024-04-27T07:04:40.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29bd972640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 0x7f29b80b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29bd972640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f29b813f7e0 0x7f29b813fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29bd972640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f29b8145ec0 0x7f29b8146350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29bd972640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f29b8140180 con 0x7f29b813f7e0 2024-04-27T07:04:40.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29bd972640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f29b8146890 con 0x7f29b8145ec0 2024-04-27T07:04:40.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29bd972640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f29b8145a00 con 0x7f29b80b4500 2024-04-27T07:04:40.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29ae7fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f29b813f7e0 0x7f29b813fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29ae7fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f29b813f7e0 0x7f29b813fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54824/0 (socket says 172.21.15.12:54824) 2024-04-27T07:04:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29ae7fc640 1 -- 172.21.15.12:0/1121291373 learned_addr learned my addr 172.21.15.12:0/1121291373 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29ae7fc640 1 -- 172.21.15.12:0/1121291373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 msgr2=0x7f29b80b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29b77fe640 1 --2- 172.21.15.12:0/1121291373 >> v2:172.21.15.53:3300/0 conn(0x7f29b8145ec0 0x7f29b8146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29b6ffd640 1 --2- 172.21.15.12:0/1121291373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 0x7f29b80b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29ae7fc640 1 --2- 172.21.15.12:0/1121291373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 0x7f29b80b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29ae7fc640 1 -- 172.21.15.12:0/1121291373 >> v2:172.21.15.53:3300/0 conn(0x7f29b8145ec0 msgr2=0x7f29b8146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29ae7fc640 1 --2- 172.21.15.12:0/1121291373 >> v2:172.21.15.53:3300/0 conn(0x7f29b8145ec0 0x7f29b8146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29ae7fc640 1 -- 172.21.15.12:0/1121291373 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b80b52b0 con 0x7f29b813f7e0 2024-04-27T07:04:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29b6ffd640 1 --2- 172.21.15.12:0/1121291373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 0x7f29b80b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:40.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.701+0000 7f29ae7fc640 1 --2- 172.21.15.12:0/1121291373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b813f7e0 0x7f29b813fc40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f29b806c9d0 tx=0x7f29a001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf3df44a9c1457c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:40.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b67fc640 1 -- 172.21.15.12:0/1121291373 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00b0070 con 0x7f29b813f7e0 2024-04-27T07:04:40.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b67fc640 1 -- 172.21.15.12:0/1121291373 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a00ab600 con 0x7f29b813f7e0 2024-04-27T07:04:40.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b67fc640 1 -- 172.21.15.12:0/1121291373 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00ab9d0 con 0x7f29b813f7e0 2024-04-27T07:04:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- 172.21.15.12:0/1121291373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b813f7e0 msgr2=0x7f29b813fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 --2- 172.21.15.12:0/1121291373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b813f7e0 0x7f29b813fc40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f29b806c9d0 tx=0x7f29a001c520 comp rx=0 tx=0).stop 2024-04-27T07:04:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- 172.21.15.12:0/1121291373 shutdown_connections 2024-04-27T07:04:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 --2- 172.21.15.12:0/1121291373 >> v2:172.21.15.53:3300/0 conn(0x7f29b8145ec0 0x7f29b8146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 --2- 172.21.15.12:0/1121291373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b813f7e0 0x7f29b813fc40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 --2- 172.21.15.12:0/1121291373 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 0x7f29b80b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- 172.21.15.12:0/1121291373 >> 172.21.15.12:0/1121291373 conn(0x7f29b80bb4f0 msgr2=0x7f29b80bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:40.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- 172.21.15.12:0/1121291373 shutdown_connections 2024-04-27T07:04:40.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- 172.21.15.12:0/1121291373 wait complete. 2024-04-27T07:04:40.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 Processor -- start 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- start start 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 0x7f29b8139020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b8145ec0 0x7f29b81e1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b81e2490 0x7f29b8140310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f29b8149b90 con 0x7f29b81e2490 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29b8149a10 con 0x7f29b8145ec0 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f29b8149d10 con 0x7f29b80b4500 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b81e2490 0x7f29b8140310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b81e2490 0x7f29b8140310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54830/0 (socket says 172.21.15.12:54830) 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b77fe640 1 -- 172.21.15.12:0/2893319741 learned_addr learned my addr 172.21.15.12:0/2893319741 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b6ffd640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 0x7f29b8139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29ae7fc640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b8145ec0 0x7f29b81e1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b77fe640 1 -- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 msgr2=0x7f29b8139020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b77fe640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 0x7f29b8139020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b77fe640 1 -- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b8145ec0 msgr2=0x7f29b81e1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b77fe640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b8145ec0 0x7f29b81e1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b77fe640 1 -- 172.21.15.12:0/2893319741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b8139de0 con 0x7f29b81e2490 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b6ffd640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 0x7f29b8139020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29ae7fc640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b8145ec0 0x7f29b81e1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b77fe640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b81e2490 0x7f29b8140310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f29b000cc30 tx=0x7f29b000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b4ff9640 1 -- 172.21.15.12:0/2893319741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b000ce40 con 0x7f29b81e2490 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b4ff9640 1 -- 172.21.15.12:0/2893319741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b00039a0 con 0x7f29b81e2490 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- 172.21.15.12:0/2893319741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b8143360 con 0x7f29b81e2490 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29b4ff9640 1 -- 172.21.15.12:0/2893319741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0003d50 con 0x7f29b81e2490 2024-04-27T07:04:40.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.705+0000 7f29bd972640 1 -- 172.21.15.12:0/2893319741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b8142da0 con 0x7f29b81e2490 2024-04-27T07:04:40.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.709+0000 7f29bd972640 1 -- 172.21.15.12:0/2893319741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b8069330 con 0x7f29b81e2490 2024-04-27T07:04:40.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.709+0000 7f29b4ff9640 1 -- 172.21.15.12:0/2893319741 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f29b0003f70 con 0x7f29b81e2490 2024-04-27T07:04:40.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.709+0000 7f29b4ff9640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2998068380 0x7f299806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:40.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.709+0000 7f29b4ff9640 1 -- 172.21.15.12:0/2893319741 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f29b0086e10 con 0x7f29b81e2490 2024-04-27T07:04:40.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.709+0000 7f29b6ffd640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2998068380 0x7f299806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:40.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.709+0000 7f29b6ffd640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2998068380 0x7f299806a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f29a8006b20 tx=0x7f29a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:40.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.713+0000 7f29b4ff9640 1 -- 172.21.15.12:0/2893319741 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29b0014030 con 0x7f29b81e2490 2024-04-27T07:04:40.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.869+0000 7f29bd972640 1 -- 172.21.15.12:0/2893319741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29b813baf0 con 0x7f29b81e2490 2024-04-27T07:04:40.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29b4ff9640 1 -- 172.21.15.12:0/2893319741 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f29b0057460 con 0x7f29b81e2490 2024-04-27T07:04:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:40.877 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 -- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2998068380 msgr2=0x7f299806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2998068380 0x7f299806a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f29a8006b20 tx=0x7f29a8008040 comp rx=0 tx=0).stop 2024-04-27T07:04:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 -- 172.21.15.12:0/2893319741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b81e2490 msgr2=0x7f29b8140310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b81e2490 0x7f29b8140310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f29b000cc30 tx=0x7f29b000a5c0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 -- 172.21.15.12:0/2893319741 shutdown_connections 2024-04-27T07:04:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2998068380 0x7f299806a840 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29b81e2490 0x7f29b8140310 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29b8145ec0 0x7f29b81e1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 --2- 172.21.15.12:0/2893319741 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f29b80b4500 0x7f29b8139020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 -- 172.21.15.12:0/2893319741 >> 172.21.15.12:0/2893319741 conn(0x7f29b80bb4f0 msgr2=0x7f29b80b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 -- 172.21.15.12:0/2893319741 shutdown_connections 2024-04-27T07:04:40.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:40.873+0000 7f29bd972640 1 -- 172.21.15.12:0/2893319741 wait complete. 2024-04-27T07:04:40.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:12.742740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:13.912898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:40.895 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:41.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a6e56640 1 Processor -- start 2024-04-27T07:04:41.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a6e56640 1 -- start start 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a6e56640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08a006e370 0x7f08a013faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a6e56640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f08a006d300 0x7f08a0150420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a6e56640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a00b4d70 con 0x7f08a00b3650 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a6e56640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f08a00740d0 con 0x7f08a006d300 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a6e56640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f08a00b2bf0 con 0x7f08a006e370 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f089ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f08a00b3650 0x7f08a00b5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43148/0 (socket says 172.21.15.12:43148) 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f089ffff640 1 -- 172.21.15.12:0/697446982 learned_addr learned my addr 172.21.15.12:0/697446982 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a53f3640 1 --2- 172.21.15.12:0/697446982 >> v2:172.21.15.12:3300/0 conn(0x7f08a006d300 0x7f08a0150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a4bf2640 1 --2- 172.21.15.12:0/697446982 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08a006e370 0x7f08a013faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f089f7fe640 1 -- 172.21.15.12:0/697446982 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 118208023 0 0) 0x7f08a00b4d70 con 0x7f08a00b3650 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f089f7fe640 1 -- 172.21.15.12:0/697446982 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0888003530 con 0x7f08a00b3650 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a53f3640 1 -- 172.21.15.12:0/697446982 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08a006e370 msgr2=0x7f08a013faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a53f3640 1 --2- 172.21.15.12:0/697446982 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08a006e370 0x7f08a013faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a53f3640 1 -- 172.21.15.12:0/697446982 >> v1:172.21.15.12:6789/0 conn(0x7f08a00b3650 legacy=0x7f08a00b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.009+0000 7f08a53f3640 1 -- 172.21.15.12:0/697446982 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a0073e60 con 0x7f08a006d300 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a53f3640 1 --2- 172.21.15.12:0/697446982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006d300 0x7f08a0150420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0898009d90 tx=0x7f089801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8284edfdc40aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089f7fe640 1 -- 172.21.15.12:0/697446982 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980b2070 con 0x7f08a006d300 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089f7fe640 1 -- 172.21.15.12:0/697446982 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08980a5850 con 0x7f08a006d300 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089f7fe640 1 -- 172.21.15.12:0/697446982 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980a5c20 con 0x7f08a006d300 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 -- 172.21.15.12:0/697446982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006d300 msgr2=0x7f08a0150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 --2- 172.21.15.12:0/697446982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006d300 0x7f08a0150420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0898009d90 tx=0x7f089801c8e0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 -- 172.21.15.12:0/697446982 shutdown_connections 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 --2- 172.21.15.12:0/697446982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006d300 0x7f08a0150420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 --2- 172.21.15.12:0/697446982 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08a006e370 0x7f08a013faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 -- 172.21.15.12:0/697446982 >> 172.21.15.12:0/697446982 conn(0x7f08a00b93b0 msgr2=0x7f08a00b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 -- 172.21.15.12:0/697446982 shutdown_connections 2024-04-27T07:04:41.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 -- 172.21.15.12:0/697446982 wait complete. 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 Processor -- start 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 -- start start 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a006d300 0x7f08a0145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006e370 0x7f08a01dea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08a00b3650 0x7f08a01def60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f08a0153510 con 0x7f08a006e370 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f08a0153390 con 0x7f08a006d300 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a6e56640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f08a0153690 con 0x7f08a00b3650 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006e370 0x7f08a01dea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006e370 0x7f08a01dea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54848/0 (socket says 172.21.15.12:54848) 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089ffff640 1 -- 172.21.15.12:0/2178475297 learned_addr learned my addr 172.21.15.12:0/2178475297 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:41.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a53f3640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08a00b3650 0x7f08a01def60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f08a4bf2640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a006d300 0x7f08a0145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089ffff640 1 -- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08a00b3650 msgr2=0x7f08a01def60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089ffff640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08a00b3650 0x7f08a01def60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089ffff640 1 -- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a006d300 msgr2=0x7f08a0145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089ffff640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a006d300 0x7f08a0145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.013+0000 7f089ffff640 1 -- 172.21.15.12:0/2178475297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a01475d0 con 0x7f08a006e370 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f089ffff640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006e370 0x7f08a01dea20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f089400c390 tx=0x7f089400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f089d7fa640 1 -- 172.21.15.12:0/2178475297 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894017070 con 0x7f08a006e370 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f089d7fa640 1 -- 172.21.15.12:0/2178475297 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0894005d10 con 0x7f08a006e370 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f089d7fa640 1 -- 172.21.15.12:0/2178475297 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894007310 con 0x7f08a006e370 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f08a6e56640 1 -- 172.21.15.12:0/2178475297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a0146c60 con 0x7f08a006e370 2024-04-27T07:04:41.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f08a6e56640 1 -- 172.21.15.12:0/2178475297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a0146030 con 0x7f08a006e370 2024-04-27T07:04:41.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f08a6e56640 1 -- 172.21.15.12:0/2178475297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a0068c20 con 0x7f08a006e370 2024-04-27T07:04:41.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f089d7fa640 1 -- 172.21.15.12:0/2178475297 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f089401a070 con 0x7f08a006e370 2024-04-27T07:04:41.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f089d7fa640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f088c068380 0x7f088c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f089d7fa640 1 -- 172.21.15.12:0/2178475297 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0894085d40 con 0x7f08a006e370 2024-04-27T07:04:41.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f08a4bf2640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f088c068380 0x7f088c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.017+0000 7f08a4bf2640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f088c068380 0x7f088c06a840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f089000b9d0 tx=0x7f0890008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:41.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.021+0000 7f089d7fa640 1 -- 172.21.15.12:0/2178475297 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0894056450 con 0x7f08a006e370 2024-04-27T07:04:41.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.181+0000 7f08a6e56640 1 -- 172.21.15.12:0/2178475297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08a01478b0 con 0x7f08a006e370 2024-04-27T07:04:41.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.181+0000 7f089d7fa640 1 -- 172.21.15.12:0/2178475297 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0894055d50 con 0x7f08a006e370 2024-04-27T07:04:41.186 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.181+0000 7f08a6e56640 1 -- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f088c068380 msgr2=0x7f088c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f088c068380 0x7f088c06a840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f089000b9d0 tx=0x7f0890008040 comp rx=0 tx=0).stop 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 -- 172.21.15.12:0/2178475297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006e370 msgr2=0x7f08a01dea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006e370 0x7f08a01dea20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f089400c390 tx=0x7f089400c850 comp rx=0 tx=0).stop 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 -- 172.21.15.12:0/2178475297 shutdown_connections 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f088c068380 0x7f088c06a840 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08a00b3650 0x7f08a01def60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08a006e370 0x7f08a01dea20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 --2- 172.21.15.12:0/2178475297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08a006d300 0x7f08a0145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 -- 172.21.15.12:0/2178475297 >> 172.21.15.12:0/2178475297 conn(0x7f08a00b93b0 msgr2=0x7f08a0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 -- 172.21.15.12:0/2178475297 shutdown_connections 2024-04-27T07:04:41.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.185+0000 7f08a6e56640 1 -- 172.21.15.12:0/2178475297 wait complete. 2024-04-27T07:04:41.204 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:04:41.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d76010640 1 Processor -- start 2024-04-27T07:04:41.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d76010640 1 -- start start 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d76010640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5d700b9d50 0x7f5d700ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d76010640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d700b7e30 0x7f5d70147f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d76010640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d700b9360 con 0x7f5d701484d0 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d76010640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f5d7014bf30 con 0x7f5d700b7e30 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d76010640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5d7014c3c0 con 0x7f5d700b9d50 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d6effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d700b7e30 0x7f5d70147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d6effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d700b7e30 0x7f5d70147f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54858/0 (socket says 172.21.15.12:54858) 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d6effd640 1 -- 172.21.15.12:0/3034755520 learned_addr learned my addr 172.21.15.12:0/3034755520 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d6f7fe640 1 --2- 172.21.15.12:0/3034755520 >> v2:172.21.15.53:3300/0 conn(0x7f5d700b9d50 0x7f5d700ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d6effd640 1 -- 172.21.15.12:0/3034755520 >> v2:172.21.15.53:3300/0 conn(0x7f5d700b9d50 msgr2=0x7f5d700ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:41.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d6effd640 1 --2- 172.21.15.12:0/3034755520 >> v2:172.21.15.53:3300/0 conn(0x7f5d700b9d50 0x7f5d700ba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d6effd640 1 -- 172.21.15.12:0/3034755520 >> v1:172.21.15.53:6789/0 conn(0x7f5d701484d0 legacy=0x7f5d700b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.325+0000 7f5d6effd640 1 -- 172.21.15.12:0/3034755520 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7014b8e0 con 0x7f5d700b7e30 2024-04-27T07:04:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d6effd640 1 --2- 172.21.15.12:0/3034755520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b7e30 0x7f5d70147f90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d70071b70 tx=0x7f5d6401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d673e6734c814c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d6e7fc640 1 -- 172.21.15.12:0/3034755520 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d640b0070 con 0x7f5d700b7e30 2024-04-27T07:04:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d6e7fc640 1 -- 172.21.15.12:0/3034755520 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d640ab600 con 0x7f5d700b7e30 2024-04-27T07:04:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d6e7fc640 1 -- 172.21.15.12:0/3034755520 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d640ab9d0 con 0x7f5d700b7e30 2024-04-27T07:04:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d76010640 1 -- 172.21.15.12:0/3034755520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b7e30 msgr2=0x7f5d70147f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d76010640 1 --2- 172.21.15.12:0/3034755520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b7e30 0x7f5d70147f90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d70071b70 tx=0x7f5d6401c3c0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d76010640 1 -- 172.21.15.12:0/3034755520 shutdown_connections 2024-04-27T07:04:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d76010640 1 --2- 172.21.15.12:0/3034755520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b7e30 0x7f5d70147f90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d76010640 1 --2- 172.21.15.12:0/3034755520 >> v2:172.21.15.53:3300/0 conn(0x7f5d700b9d50 0x7f5d700ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d76010640 1 -- 172.21.15.12:0/3034755520 >> 172.21.15.12:0/3034755520 conn(0x7f5d700ba6d0 msgr2=0x7f5d700baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d76010640 1 -- 172.21.15.12:0/3034755520 shutdown_connections 2024-04-27T07:04:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.329+0000 7f5d76010640 1 -- 172.21.15.12:0/3034755520 wait complete. 2024-04-27T07:04:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 Processor -- start 2024-04-27T07:04:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 -- start start 2024-04-27T07:04:41.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d700b7e30 0x7f5d7014ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b9d50 0x7f5d701408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5d701484d0 0x7f5d701e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701551d0 con 0x7f5d700b9d50 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70155050 con 0x7f5d700b7e30 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f5d70155350 con 0x7f5d701484d0 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b9d50 0x7f5d701408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b9d50 0x7f5d701408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54862/0 (socket says 172.21.15.12:54862) 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6effd640 1 -- 172.21.15.12:0/3760869745 learned_addr learned my addr 172.21.15.12:0/3760869745 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6f7fe640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d700b7e30 0x7f5d7014ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6ffff640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5d701484d0 0x7f5d701e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6effd640 1 -- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5d701484d0 msgr2=0x7f5d701e36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6effd640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5d701484d0 0x7f5d701e36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6effd640 1 -- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d700b7e30 msgr2=0x7f5d7014ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6effd640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d700b7e30 0x7f5d7014ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6effd640 1 -- 172.21.15.12:0/3760869745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7009d6e0 con 0x7f5d700b9d50 2024-04-27T07:04:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d6effd640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b9d50 0x7f5d701408b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5d6401aa70 tx=0x7f5d64019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d57fff640 1 -- 172.21.15.12:0/3760869745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d640b0040 con 0x7f5d700b9d50 2024-04-27T07:04:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d57fff640 1 -- 172.21.15.12:0/3760869745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d640b9070 con 0x7f5d700b9d50 2024-04-27T07:04:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d57fff640 1 -- 172.21.15.12:0/3760869745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d640b4680 con 0x7f5d700b9d50 2024-04-27T07:04:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 -- 172.21.15.12:0/3760869745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7009e010 con 0x7f5d700b9d50 2024-04-27T07:04:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 -- 172.21.15.12:0/3760869745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7009e5d0 con 0x7f5d700b9d50 2024-04-27T07:04:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d76010640 1 -- 172.21.15.12:0/3760869745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d70069330 con 0x7f5d700b9d50 2024-04-27T07:04:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.333+0000 7f5d57fff640 1 -- 172.21.15.12:0/3760869745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5d640b4820 con 0x7f5d700b9d50 2024-04-27T07:04:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.337+0000 7f5d57fff640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5d5c068380 0x7f5d5c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.337+0000 7f5d57fff640 1 -- 172.21.15.12:0/3760869745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5d64120480 con 0x7f5d700b9d50 2024-04-27T07:04:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.337+0000 7f5d6f7fe640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5d5c068380 0x7f5d5c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.337+0000 7f5d6f7fe640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5d5c068380 0x7f5d5c06a840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d60007a60 tx=0x7f5d60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:41.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.341+0000 7f5d57fff640 1 -- 172.21.15.12:0/3760869745 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d640160d0 con 0x7f5d700b9d50 2024-04-27T07:04:41.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.501+0000 7f5d76010640 1 -- 172.21.15.12:0/3760869745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5d7009e7a0 con 0x7f5d700b9d50 2024-04-27T07:04:41.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d57fff640 1 -- 172.21.15.12:0/3760869745 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f5d640f0480 con 0x7f5d700b9d50 2024-04-27T07:04:41.508 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:04:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 -- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5d5c068380 msgr2=0x7f5d5c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5d5c068380 0x7f5d5c06a840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5d60007a60 tx=0x7f5d60008040 comp rx=0 tx=0).stop 2024-04-27T07:04:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 -- 172.21.15.12:0/3760869745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b9d50 msgr2=0x7f5d701408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b9d50 0x7f5d701408b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5d6401aa70 tx=0x7f5d64019160 comp rx=0 tx=0).stop 2024-04-27T07:04:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 -- 172.21.15.12:0/3760869745 shutdown_connections 2024-04-27T07:04:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5d5c068380 0x7f5d5c06a840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5d701484d0 0x7f5d701e36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d700b9d50 0x7f5d701408b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 --2- 172.21.15.12:0/3760869745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d700b7e30 0x7f5d7014ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 -- 172.21.15.12:0/3760869745 >> 172.21.15.12:0/3760869745 conn(0x7f5d700ba6d0 msgr2=0x7f5d7006e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:41.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 -- 172.21.15.12:0/3760869745 shutdown_connections 2024-04-27T07:04:41.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.505+0000 7f5d76010640 1 -- 172.21.15.12:0/3760869745 wait complete. 2024-04-27T07:04:41.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:04:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7766876640 1 Processor -- start 2024-04-27T07:04:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7766876640 1 -- start start 2024-04-27T07:04:41.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7766876640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7760067b70 0x7f7760143120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7766876640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f77600b31f0 0x7f77601504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7766876640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7760068870 con 0x7f7760143660 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7766876640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f77600b2650 con 0x7f77600b31f0 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7766876640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7760067f50 con 0x7f7760067b70 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7764e13640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f77600b31f0 0x7f77601504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f775f7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7760143660 0x7f77600b2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43152/0 (socket says 172.21.15.12:43152) 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f775f7fe640 1 -- 172.21.15.12:0/1384203998 learned_addr learned my addr 172.21.15.12:0/1384203998 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f775ffff640 1 --2- 172.21.15.12:0/1384203998 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7760067b70 0x7f7760143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7764e13640 1 -- 172.21.15.12:0/1384203998 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7760067b70 msgr2=0x7f7760143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7764e13640 1 --2- 172.21.15.12:0/1384203998 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7760067b70 0x7f7760143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7764e13640 1 -- 172.21.15.12:0/1384203998 >> v1:172.21.15.12:6789/0 conn(0x7f7760143660 legacy=0x7f77600b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f7764e13640 1 -- 172.21.15.12:0/1384203998 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7760072f70 con 0x7f77600b31f0 2024-04-27T07:04:41.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.653+0000 7f775effd640 1 -- 172.21.15.12:0/1384203998 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3597422246 0 0) 0x7f7760068870 con 0x7f7760143660 2024-04-27T07:04:41.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7764e13640 1 --2- 172.21.15.12:0/1384203998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77600b31f0 0x7f77601504e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7758009d90 tx=0x7f775801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd405bf843ade0b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:41.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f775effd640 1 -- 172.21.15.12:0/1384203998 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580b2070 con 0x7f77600b31f0 2024-04-27T07:04:41.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f775effd640 1 -- 172.21.15.12:0/1384203998 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77580a58b0 con 0x7f77600b31f0 2024-04-27T07:04:41.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f775effd640 1 -- 172.21.15.12:0/1384203998 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580a5c80 con 0x7f77600b31f0 2024-04-27T07:04:41.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 -- 172.21.15.12:0/1384203998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77600b31f0 msgr2=0x7f77601504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 --2- 172.21.15.12:0/1384203998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77600b31f0 0x7f77601504e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7758009d90 tx=0x7f775801c8e0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 -- 172.21.15.12:0/1384203998 shutdown_connections 2024-04-27T07:04:41.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 --2- 172.21.15.12:0/1384203998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77600b31f0 0x7f77601504e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 --2- 172.21.15.12:0/1384203998 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7760067b70 0x7f7760143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 -- 172.21.15.12:0/1384203998 >> 172.21.15.12:0/1384203998 conn(0x7f77600b67b0 msgr2=0x7f77600b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:41.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 -- 172.21.15.12:0/1384203998 shutdown_connections 2024-04-27T07:04:41.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 -- 172.21.15.12:0/1384203998 wait complete. 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 Processor -- start 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 -- start start 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7760067b70 0x7f776014f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77600b31f0 0x7f77601e0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7760143660 0x7f77601e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f77600b72d0 con 0x7f7760143660 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f77600b3a90 con 0x7f77600b31f0 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7766876640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f77600b7450 con 0x7f7760067b70 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7764e13640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7760143660 0x7f77601e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7764e13640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7760143660 0x7f77601e0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54886/0 (socket says 172.21.15.12:54886) 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7764e13640 1 -- 172.21.15.12:0/2984115539 learned_addr learned my addr 172.21.15.12:0/2984115539 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f775ffff640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7760067b70 0x7f776014f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f775f7fe640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77600b31f0 0x7f77601e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7764e13640 1 -- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7760067b70 msgr2=0x7f776014f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7764e13640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7760067b70 0x7f776014f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7764e13640 1 -- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77600b31f0 msgr2=0x7f77601e0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7764e13640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77600b31f0 0x7f77601e0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7764e13640 1 -- 172.21.15.12:0/2984115539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776013ede0 con 0x7f7760143660 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f7764e13640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7760143660 0x7f77601e0ba0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f77580b0040 tx=0x7f7758019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.657+0000 7f775cff9640 1 -- 172.21.15.12:0/2984115539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580b2050 con 0x7f7760143660 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.661+0000 7f775cff9640 1 -- 172.21.15.12:0/2984115539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77580be070 con 0x7f7760143660 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.661+0000 7f7766876640 1 -- 172.21.15.12:0/2984115539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776013f0c0 con 0x7f7760143660 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.661+0000 7f775cff9640 1 -- 172.21.15.12:0/2984115539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580ada90 con 0x7f7760143660 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.661+0000 7f7766876640 1 -- 172.21.15.12:0/2984115539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7760146980 con 0x7f7760143660 2024-04-27T07:04:41.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.661+0000 7f7766876640 1 -- 172.21.15.12:0/2984115539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7760069330 con 0x7f7760143660 2024-04-27T07:04:41.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.661+0000 7f775cff9640 1 -- 172.21.15.12:0/2984115539 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f77580adc30 con 0x7f7760143660 2024-04-27T07:04:41.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.661+0000 7f775cff9640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f774c068380 0x7f774c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:41.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.661+0000 7f775cff9640 1 -- 172.21.15.12:0/2984115539 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7758121a20 con 0x7f7760143660 2024-04-27T07:04:41.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.661+0000 7f775ffff640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f774c068380 0x7f774c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:41.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.661+0000 7f775ffff640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f774c068380 0x7f774c06a840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f775000b930 tx=0x7f7750008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:41.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.665+0000 7f775cff9640 1 -- 172.21.15.12:0/2984115539 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7758016560 con 0x7f7760143660 2024-04-27T07:04:41.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.825+0000 7f7766876640 1 -- 172.21.15.12:0/2984115539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f776013eb00 con 0x7f7760143660 2024-04-27T07:04:41.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.825+0000 7f775cff9640 1 -- 172.21.15.12:0/2984115539 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f77580ab340 con 0x7f7760143660 2024-04-27T07:04:41.830 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:04:41.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 -- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f774c068380 msgr2=0x7f774c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f774c068380 0x7f774c06a840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f775000b930 tx=0x7f7750008040 comp rx=0 tx=0).stop 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 -- 172.21.15.12:0/2984115539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7760143660 msgr2=0x7f77601e0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7760143660 0x7f77601e0ba0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f77580b0040 tx=0x7f7758019680 comp rx=0 tx=0).stop 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 -- 172.21.15.12:0/2984115539 shutdown_connections 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f774c068380 0x7f774c06a840 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7760143660 0x7f77601e0ba0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f77600b31f0 0x7f77601e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 --2- 172.21.15.12:0/2984115539 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7760067b70 0x7f776014f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 -- 172.21.15.12:0/2984115539 >> 172.21.15.12:0/2984115539 conn(0x7f77600b67b0 msgr2=0x7f77600b89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 -- 172.21.15.12:0/2984115539 shutdown_connections 2024-04-27T07:04:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:41.829+0000 7f7766876640 1 -- 172.21.15.12:0/2984115539 wait complete. 2024-04-27T07:04:41.912 INFO:tasks.cephfs_test_runner:Starting test: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) Test that when cd is issued with an argument, CWD is changed 2024-04-27T07:04:41.912 INFO:tasks.cephfs_test_runner:test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-27T07:04:41.913 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:42.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f293b06f640 1 Processor -- start 2024-04-27T07:04:42.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f293b06f640 1 -- start start 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f293b06f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2934148310 0x7f293414b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f293b06f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2934141620 0x7f2934141a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f293b06f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29340b7990 con 0x7f29340baa00 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f293b06f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f29340b73d0 con 0x7f2934148310 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f293b06f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f29340bade0 con 0x7f2934141620 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f2933fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2934148310 0x7f293414b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f293960c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f2934141620 0x7f2934141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f2933fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2934148310 0x7f293414b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54890/0 (socket says 172.21.15.12:54890) 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f2933fff640 1 -- 172.21.15.12:0/2801737661 learned_addr learned my addr 172.21.15.12:0/2801737661 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f2933fff640 1 -- 172.21.15.12:0/2801737661 >> v2:172.21.15.53:3300/0 conn(0x7f2934141620 msgr2=0x7f2934141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f2933fff640 1 --2- 172.21.15.12:0/2801737661 >> v2:172.21.15.53:3300/0 conn(0x7f2934141620 0x7f2934141a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f2933fff640 1 -- 172.21.15.12:0/2801737661 >> v1:172.21.15.53:6789/0 conn(0x7f29340baa00 legacy=0x7f2934147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f2933fff640 1 -- 172.21.15.12:0/2801737661 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29340b76b0 con 0x7f2934148310 2024-04-27T07:04:42.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.033+0000 7f2933fff640 1 --2- 172.21.15.12:0/2801737661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 0x7f293414b1f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29341457d0 tx=0x7f292001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ea5d546f83fdaa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f29337fe640 1 -- 172.21.15.12:0/2801737661 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200b1070 con 0x7f2934148310 2024-04-27T07:04:42.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f29337fe640 1 -- 172.21.15.12:0/2801737661 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29200ac630 con 0x7f2934148310 2024-04-27T07:04:42.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f29337fe640 1 -- 172.21.15.12:0/2801737661 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200a8050 con 0x7f2934148310 2024-04-27T07:04:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- 172.21.15.12:0/2801737661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 msgr2=0x7f293414b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 --2- 172.21.15.12:0/2801737661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 0x7f293414b1f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29341457d0 tx=0x7f292001c3d0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- 172.21.15.12:0/2801737661 shutdown_connections 2024-04-27T07:04:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 --2- 172.21.15.12:0/2801737661 >> v2:172.21.15.53:3300/0 conn(0x7f2934141620 0x7f2934141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 --2- 172.21.15.12:0/2801737661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 0x7f293414b1f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- 172.21.15.12:0/2801737661 >> 172.21.15.12:0/2801737661 conn(0x7f29340738e0 msgr2=0x7f29340ba090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- 172.21.15.12:0/2801737661 shutdown_connections 2024-04-27T07:04:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- 172.21.15.12:0/2801737661 wait complete. 2024-04-27T07:04:42.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 Processor -- start 2024-04-27T07:04:42.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- start start 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29340baa00 0x7f2934149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2934141620 0x7f29341e0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 0x7f29341e09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2934141fb0 con 0x7f2934148310 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2934154da0 con 0x7f29340baa00 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f2934144f20 con 0x7f2934141620 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293960c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 0x7f29341e09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293960c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 0x7f29341e09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54896/0 (socket says 172.21.15.12:54896) 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293960c640 1 -- 172.21.15.12:0/1051860168 learned_addr learned my addr 172.21.15.12:0/1051860168 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f2938e0b640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29340baa00 0x7f2934149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f2933fff640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2934141620 0x7f29341e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293960c640 1 -- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2934141620 msgr2=0x7f29341e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293960c640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2934141620 0x7f29341e0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293960c640 1 -- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29340baa00 msgr2=0x7f2934149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293960c640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29340baa00 0x7f2934149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293960c640 1 -- 172.21.15.12:0/1051860168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293414a6a0 con 0x7f2934148310 2024-04-27T07:04:42.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293960c640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 0x7f29341e09d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f292c009dd0 tx=0x7f292c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f29317fa640 1 -- 172.21.15.12:0/1051860168 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c018020 con 0x7f2934148310 2024-04-27T07:04:42.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f29317fa640 1 -- 172.21.15.12:0/1051860168 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292c014480 con 0x7f2934148310 2024-04-27T07:04:42.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- 172.21.15.12:0/1051860168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293414ad40 con 0x7f2934148310 2024-04-27T07:04:42.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f29317fa640 1 -- 172.21.15.12:0/1051860168 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c014870 con 0x7f2934148310 2024-04-27T07:04:42.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.037+0000 7f293b06f640 1 -- 172.21.15.12:0/1051860168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293414a3c0 con 0x7f2934148310 2024-04-27T07:04:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.041+0000 7f293b06f640 1 -- 172.21.15.12:0/1051860168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29341496d0 con 0x7f2934148310 2024-04-27T07:04:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.041+0000 7f29317fa640 1 -- 172.21.15.12:0/1051860168 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f292c01b020 con 0x7f2934148310 2024-04-27T07:04:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.041+0000 7f29317fa640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2918068380 0x7f291806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.041+0000 7f29317fa640 1 -- 172.21.15.12:0/1051860168 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f292c087480 con 0x7f2934148310 2024-04-27T07:04:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.041+0000 7f2938e0b640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2918068380 0x7f291806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.041+0000 7f2938e0b640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2918068380 0x7f291806a840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2928001650 tx=0x7f2928009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.045+0000 7f29317fa640 1 -- 172.21.15.12:0/1051860168 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29341496d0 con 0x7f2934148310 2024-04-27T07:04:42.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.201+0000 7f293b06f640 1 -- 172.21.15.12:0/1051860168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f293414ab30 con 0x7f2934148310 2024-04-27T07:04:42.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.201+0000 7f29317fa640 1 -- 172.21.15.12:0/1051860168 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f292c022070 con 0x7f2934148310 2024-04-27T07:04:42.207 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:42.208 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 -- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2918068380 msgr2=0x7f291806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2918068380 0x7f291806a840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2928001650 tx=0x7f2928009040 comp rx=0 tx=0).stop 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 -- 172.21.15.12:0/1051860168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 msgr2=0x7f29341e09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 0x7f29341e09d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f292c009dd0 tx=0x7f292c00c490 comp rx=0 tx=0).stop 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 -- 172.21.15.12:0/1051860168 shutdown_connections 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2918068380 0x7f291806a840 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2934148310 0x7f29341e09d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f2934141620 0x7f29341e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 --2- 172.21.15.12:0/1051860168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29340baa00 0x7f2934149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 -- 172.21.15.12:0/1051860168 >> 172.21.15.12:0/1051860168 conn(0x7f29340738e0 msgr2=0x7f29340b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 -- 172.21.15.12:0/1051860168 shutdown_connections 2024-04-27T07:04:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.205+0000 7f293b06f640 1 -- 172.21.15.12:0/1051860168 wait complete. 2024-04-27T07:04:42.226 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:12.742740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:13.912898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:42.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f065d6ff640 1 Processor -- start 2024-04-27T07:04:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f065d6ff640 1 -- start start 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f065d6ff640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f065806d300 0x7f0658151430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f065d6ff640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06580b4d70 con 0x7f06580b3650 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f065d6ff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0658073e60 con 0x7f065806e370 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f065d6ff640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f06580740d0 con 0x7f065806d300 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06567fc640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f06580b3650 0x7f06580b5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43160/0 (socket says 172.21.15.12:43160) 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06567fc640 1 -- 172.21.15.12:0/1710116845 learned_addr learned my addr 172.21.15.12:0/1710116845 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06577fe640 1 --2- 172.21.15.12:0/1710116845 >> v2:172.21.15.53:3300/0 conn(0x7f065806d300 0x7f0658151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1981933793 0 0) 0x7f06580b4d70 con 0x7f06580b3650 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0640003530 con 0x7f06580b3650 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4195216201 0 0) 0x7f0640003530 con 0x7f06580b3650 2024-04-27T07:04:42.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 >> v2:172.21.15.53:3300/0 conn(0x7f065806d300 msgr2=0x7f0658151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 --2- 172.21.15.12:0/1710116845 >> v2:172.21.15.53:3300/0 conn(0x7f065806d300 0x7f0658151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 >> v1:172.21.15.53:6789/0 conn(0x7f065806e370 legacy=0x7f065813faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06580b2bf0 con 0x7f06580b3650 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f064c001680 con 0x7f06580b3650 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06577fe640 1 --2- 172.21.15.12:0/1710116845 >> v2:172.21.15.53:3300/0 conn(0x7f065806d300 0x7f0658151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 >> v1:172.21.15.12:6789/0 conn(0x7f06580b3650 legacy=0x7f06580b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400039c0 0x7f0640003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0640004810 0x7f0640004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06400052f0 0x7f064000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f06580740d0 con 0x7f0640004810 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f064000c1c0 con 0x7f06400052f0 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f064000c3c0 con 0x7f06400039c0 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06567fc640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0640004810 0x7f0640004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06577fe640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06400052f0 0x7f064000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f0656ffd640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400039c0 0x7f0640003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06567fc640 1 -- 172.21.15.12:0/1710116845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400039c0 msgr2=0x7f0640003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:42.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06567fc640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f06400039c0 0x7f0640003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06567fc640 1 -- 172.21.15.12:0/1710116845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06400052f0 msgr2=0x7f064000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:42.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06567fc640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06400052f0 0x7f064000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06577fe640 1 -- 172.21.15.12:0/1710116845 reap_dead start 2024-04-27T07:04:42.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.345+0000 7f06567fc640 1 -- 172.21.15.12:0/1710116845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0640003530 con 0x7f0640004810 2024-04-27T07:04:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06567fc640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0640004810 0x7f0640004c90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f06580b3a70 tx=0x7f064c01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35dd763f65876768 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c007d00 con 0x7f0640004810 2024-04-27T07:04:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f064c0a9020 con 0x7f0640004810 2024-04-27T07:04:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f0655ffb640 1 -- 172.21.15.12:0/1710116845 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c005ae0 con 0x7f0640004810 2024-04-27T07:04:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 -- 172.21.15.12:0/1710116845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0640004810 msgr2=0x7f0640004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0640004810 0x7f0640004c90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f06580b3a70 tx=0x7f064c01cc00 comp rx=0 tx=0).stop 2024-04-27T07:04:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 -- 172.21.15.12:0/1710116845 shutdown_connections 2024-04-27T07:04:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 --2- 172.21.15.12:0/1710116845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0640004810 0x7f0640004c90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 -- 172.21.15.12:0/1710116845 >> 172.21.15.12:0/1710116845 conn(0x7f06580b93b0 msgr2=0x7f06580b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 -- 172.21.15.12:0/1710116845 shutdown_connections 2024-04-27T07:04:42.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 -- 172.21.15.12:0/1710116845 wait complete. 2024-04-27T07:04:42.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 Processor -- start 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 -- start start 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0640004810 0x7f065809e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06581e1c30 0x7f06581e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0658138100 0x7f0658138580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f06580b5520 con 0x7f06581e1c30 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f06580b53a0 con 0x7f0658138100 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f065d6ff640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f06580b56a0 con 0x7f0640004810 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06567fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06581e1c30 0x7f06581e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06567fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06581e1c30 0x7f06581e20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54916/0 (socket says 172.21.15.12:54916) 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06567fc640 1 -- 172.21.15.12:0/2889912024 learned_addr learned my addr 172.21.15.12:0/2889912024 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06577fe640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0658138100 0x7f0658138580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f0656ffd640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0640004810 0x7f065809e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06567fc640 1 -- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0640004810 msgr2=0x7f065809e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06567fc640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0640004810 0x7f065809e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06567fc640 1 -- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0658138100 msgr2=0x7f0658138580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06567fc640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0658138100 0x7f0658138580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06567fc640 1 -- 172.21.15.12:0/2889912024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065809cb70 con 0x7f06581e1c30 2024-04-27T07:04:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06577fe640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0658138100 0x7f0658138580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.349+0000 7f06567fc640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06581e1c30 0x7f06581e20b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f064c0ab040 tx=0x7f0640005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f063f7fe640 1 -- 172.21.15.12:0/2889912024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c0b4070 con 0x7f06581e1c30 2024-04-27T07:04:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f063f7fe640 1 -- 172.21.15.12:0/2889912024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f064c000ee0 con 0x7f06581e1c30 2024-04-27T07:04:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f063f7fe640 1 -- 172.21.15.12:0/2889912024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c007d00 con 0x7f06581e1c30 2024-04-27T07:04:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f065d6ff640 1 -- 172.21.15.12:0/2889912024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06581481f0 con 0x7f06581e1c30 2024-04-27T07:04:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f065d6ff640 1 -- 172.21.15.12:0/2889912024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0658147f10 con 0x7f06581e1c30 2024-04-27T07:04:42.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f065d6ff640 1 -- 172.21.15.12:0/2889912024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065809cd40 con 0x7f06581e1c30 2024-04-27T07:04:42.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f063f7fe640 1 -- 172.21.15.12:0/2889912024 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f064c0bf060 con 0x7f06581e1c30 2024-04-27T07:04:42.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f063f7fe640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0644068380 0x7f064406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f063f7fe640 1 -- 172.21.15.12:0/2889912024 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f064c11f5d0 con 0x7f06581e1c30 2024-04-27T07:04:42.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f0656ffd640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0644068380 0x7f064406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.353+0000 7f0656ffd640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0644068380 0x7f064406a840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0658149f00 tx=0x7f0648009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.357+0000 7f063f7fe640 1 -- 172.21.15.12:0/2889912024 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f064c016910 con 0x7f06581e1c30 2024-04-27T07:04:42.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.513+0000 7f065d6ff640 1 -- 172.21.15.12:0/2889912024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0658068c20 con 0x7f06581e1c30 2024-04-27T07:04:42.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.513+0000 7f063f7fe640 1 -- 172.21.15.12:0/2889912024 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f064c0efc20 con 0x7f06581e1c30 2024-04-27T07:04:42.519 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 -- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0644068380 msgr2=0x7f064406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0644068380 0x7f064406a840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0658149f00 tx=0x7f0648009040 comp rx=0 tx=0).stop 2024-04-27T07:04:42.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 -- 172.21.15.12:0/2889912024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06581e1c30 msgr2=0x7f06581e20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06581e1c30 0x7f06581e20b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f064c0ab040 tx=0x7f0640005840 comp rx=0 tx=0).stop 2024-04-27T07:04:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 -- 172.21.15.12:0/2889912024 shutdown_connections 2024-04-27T07:04:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0644068380 0x7f064406a840 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0658138100 0x7f0658138580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06581e1c30 0x7f06581e20b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 --2- 172.21.15.12:0/2889912024 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0640004810 0x7f065809e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 -- 172.21.15.12:0/2889912024 >> 172.21.15.12:0/2889912024 conn(0x7f06580b93b0 msgr2=0x7f065813cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 -- 172.21.15.12:0/2889912024 shutdown_connections 2024-04-27T07:04:42.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.517+0000 7f065d6ff640 1 -- 172.21.15.12:0/2889912024 wait complete. 2024-04-27T07:04:42.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:04:42.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.645+0000 7f9a3ae64640 1 Processor -- start 2024-04-27T07:04:42.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.645+0000 7f9a3ae64640 1 -- start start 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a3ae64640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a3406f140 0x7f9a3406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a3ae64640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9a34145e10 0x7f9a341462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a3ae64640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a341467e0 con 0x7f9a340b74b0 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a3ae64640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f9a3406fd30 con 0x7f9a34145e10 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a3ae64640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3406f5c0 con 0x7f9a3406f140 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a38c00640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9a340b74b0 0x7f9a340b9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43168/0 (socket says 172.21.15.12:43168) 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a38c00640 1 -- 172.21.15.12:0/2428866669 learned_addr learned my addr 172.21.15.12:0/2428866669 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a33fff640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a3406f140 0x7f9a3406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a39401640 1 --2- 172.21.15.12:0/2428866669 >> v2:172.21.15.53:3300/0 conn(0x7f9a34145e10 0x7f9a341462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432296941 0 0) 0x7f9a341467e0 con 0x7f9a340b74b0 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a1c003530 con 0x7f9a340b74b0 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 303612770 0 0) 0x7f9a1c003530 con 0x7f9a340b74b0 2024-04-27T07:04:42.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a3406f140 msgr2=0x7f9a3406ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a3406f140 0x7f9a3406ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 >> v2:172.21.15.53:3300/0 conn(0x7f9a34145e10 msgr2=0x7f9a341462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 --2- 172.21.15.12:0/2428866669 >> v2:172.21.15.53:3300/0 conn(0x7f9a34145e10 0x7f9a341462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a340ba4f0 con 0x7f9a340b74b0 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f9a24001680 con 0x7f9a340b74b0 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 >> v1:172.21.15.12:6789/0 conn(0x7f9a340b74b0 legacy=0x7f9a340b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a3406f140 0x7f9a1c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c003f40 0x7f9a1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a1c004d50 0x7f9a1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3406fd30 con 0x7f9a1c003f40 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3406f5c0 con 0x7f9a1c004d50 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c00b9d0 con 0x7f9a3406f140 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a33fff640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c003f40 0x7f9a1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a38c00640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a3406f140 0x7f9a1c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a39401640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a1c004d50 0x7f9a1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a33fff640 1 -- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a3406f140 msgr2=0x7f9a1c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:42.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a33fff640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a3406f140 0x7f9a1c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a33fff640 1 -- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a1c004d50 msgr2=0x7f9a1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:42.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a33fff640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a1c004d50 0x7f9a1c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.649+0000 7f9a33fff640 1 -- 172.21.15.12:0/2428866669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c003530 con 0x7f9a1c003f40 2024-04-27T07:04:42.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a33fff640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c003f40 0x7f9a1c004810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009ae0 tx=0x7f9a2801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc83a0a6994557f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280b2070 con 0x7f9a1c003f40 2024-04-27T07:04:42.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a280a5550 con 0x7f9a1c003f40 2024-04-27T07:04:42.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a337fe640 1 -- 172.21.15.12:0/2428866669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280096b0 con 0x7f9a1c003f40 2024-04-27T07:04:42.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- 172.21.15.12:0/2428866669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c003f40 msgr2=0x7f9a1c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 --2- 172.21.15.12:0/2428866669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c003f40 0x7f9a1c004810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009ae0 tx=0x7f9a2801ca10 comp rx=0 tx=0).stop 2024-04-27T07:04:42.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a39401640 1 -- 172.21.15.12:0/2428866669 reap_dead start 2024-04-27T07:04:42.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- 172.21.15.12:0/2428866669 shutdown_connections 2024-04-27T07:04:42.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- 172.21.15.12:0/2428866669 >> 172.21.15.12:0/2428866669 conn(0x7f9a340ba6d0 msgr2=0x7f9a340baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:42.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- 172.21.15.12:0/2428866669 shutdown_connections 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- 172.21.15.12:0/2428866669 wait complete. 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 Processor -- start 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- start start 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c003f40 0x7f9a340b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a341dec60 0x7f9a341df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a34140560 0x7f9a341409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a340b7890 con 0x7f9a34140560 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9a340ba070 con 0x7f9a341dec60 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f9a340b7a10 con 0x7f9a1c003f40 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a39401640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a34140560 0x7f9a341409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a39401640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a34140560 0x7f9a341409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54926/0 (socket says 172.21.15.12:54926) 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a39401640 1 -- 172.21.15.12:0/1571327019 learned_addr learned my addr 172.21.15.12:0/1571327019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a39401640 1 -- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c003f40 msgr2=0x7f9a340b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a39401640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c003f40 0x7f9a340b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a39401640 1 -- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a341dec60 msgr2=0x7f9a341df0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a39401640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a341dec60 0x7f9a341df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a39401640 1 -- 172.21.15.12:0/1571327019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a341df620 con 0x7f9a34140560 2024-04-27T07:04:42.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a39401640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a34140560 0x7f9a341409e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9a34070250 tx=0x7f9a2c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a317fa640 1 -- 172.21.15.12:0/1571327019 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c016620 con 0x7f9a34140560 2024-04-27T07:04:42.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a317fa640 1 -- 172.21.15.12:0/1571327019 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2c01b070 con 0x7f9a34140560 2024-04-27T07:04:42.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- 172.21.15.12:0/1571327019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a3409d7b0 con 0x7f9a34140560 2024-04-27T07:04:42.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a317fa640 1 -- 172.21.15.12:0/1571327019 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c016a20 con 0x7f9a34140560 2024-04-27T07:04:42.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.653+0000 7f9a3ae64640 1 -- 172.21.15.12:0/1571327019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a340b4770 con 0x7f9a34140560 2024-04-27T07:04:42.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.657+0000 7f9a3ae64640 1 -- 172.21.15.12:0/1571327019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a34069330 con 0x7f9a34140560 2024-04-27T07:04:42.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.657+0000 7f9a317fa640 1 -- 172.21.15.12:0/1571327019 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9a2c016bc0 con 0x7f9a34140560 2024-04-27T07:04:42.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.657+0000 7f9a317fa640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9a200683b0 0x7f9a2006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.657+0000 7f9a317fa640 1 -- 172.21.15.12:0/1571327019 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9a2c087610 con 0x7f9a34140560 2024-04-27T07:04:42.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.657+0000 7f9a38c00640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9a200683b0 0x7f9a2006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.657+0000 7f9a38c00640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9a200683b0 0x7f9a2006a870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9a24004880 tx=0x7f9a24000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.661+0000 7f9a317fa640 1 -- 172.21.15.12:0/1571327019 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9a341df620 con 0x7f9a34140560 2024-04-27T07:04:42.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.817+0000 7f9a3ae64640 1 -- 172.21.15.12:0/1571327019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9a340b7c70 con 0x7f9a34140560 2024-04-27T07:04:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.817+0000 7f9a317fa640 1 -- 172.21.15.12:0/1571327019 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f9a2c057c60 con 0x7f9a34140560 2024-04-27T07:04:42.824 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:04:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 -- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9a200683b0 msgr2=0x7f9a2006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9a200683b0 0x7f9a2006a870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9a24004880 tx=0x7f9a24000ee0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 -- 172.21.15.12:0/1571327019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a34140560 msgr2=0x7f9a341409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a34140560 0x7f9a341409e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9a34070250 tx=0x7f9a2c00f520 comp rx=0 tx=0).stop 2024-04-27T07:04:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 -- 172.21.15.12:0/1571327019 shutdown_connections 2024-04-27T07:04:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9a200683b0 0x7f9a2006a870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a34140560 0x7f9a341409e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a341dec60 0x7f9a341df0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 --2- 172.21.15.12:0/1571327019 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9a1c003f40 0x7f9a340b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 -- 172.21.15.12:0/1571327019 >> 172.21.15.12:0/1571327019 conn(0x7f9a340ba6d0 msgr2=0x7f9a34072220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:42.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 -- 172.21.15.12:0/1571327019 shutdown_connections 2024-04-27T07:04:42.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.821+0000 7f9a3ae64640 1 -- 172.21.15.12:0/1571327019 wait complete. 2024-04-27T07:04:42.842 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:04:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f826b6b5640 1 Processor -- start 2024-04-27T07:04:42.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f826b6b5640 1 -- start start 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f826b6b5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82640b69a0 0x7f8264144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f826b6b5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8264141670 0x7f8264141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f826b6b5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8264144870 con 0x7f826409d440 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f826b6b5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f8264142010 con 0x7f8264141670 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f826b6b5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f82640b58d0 con 0x7f82640b69a0 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f8268c50640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8264141670 0x7f8264141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f8269451640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f82640b69a0 0x7f8264144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f8268c50640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8264141670 0x7f8264141ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54932/0 (socket says 172.21.15.12:54932) 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f8268c50640 1 -- 172.21.15.12:0/509021464 learned_addr learned my addr 172.21.15.12:0/509021464 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f8268c50640 1 -- 172.21.15.12:0/509021464 >> v2:172.21.15.53:3300/0 conn(0x7f82640b69a0 msgr2=0x7f8264144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f8268c50640 1 --2- 172.21.15.12:0/509021464 >> v2:172.21.15.53:3300/0 conn(0x7f82640b69a0 0x7f8264144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f8268c50640 1 -- 172.21.15.12:0/509021464 >> v1:172.21.15.53:6789/0 conn(0x7f826409d440 legacy=0x7f826409d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f8268c50640 1 -- 172.21.15.12:0/509021464 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82640b6d80 con 0x7f8264141670 2024-04-27T07:04:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.953+0000 7f8268c50640 1 --2- 172.21.15.12:0/509021464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 0x7f8264141ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8264069d80 tx=0x7f825401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00a063c641a3949 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f825bfff640 1 -- 172.21.15.12:0/509021464 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82540b0070 con 0x7f8264141670 2024-04-27T07:04:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f825bfff640 1 -- 172.21.15.12:0/509021464 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82540ab600 con 0x7f8264141670 2024-04-27T07:04:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f825bfff640 1 -- 172.21.15.12:0/509021464 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82540ab9d0 con 0x7f8264141670 2024-04-27T07:04:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- 172.21.15.12:0/509021464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 msgr2=0x7f8264141ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 --2- 172.21.15.12:0/509021464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 0x7f8264141ad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8264069d80 tx=0x7f825401c470 comp rx=0 tx=0).stop 2024-04-27T07:04:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- 172.21.15.12:0/509021464 shutdown_connections 2024-04-27T07:04:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 --2- 172.21.15.12:0/509021464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 0x7f8264141ad0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 --2- 172.21.15.12:0/509021464 >> v2:172.21.15.53:3300/0 conn(0x7f82640b69a0 0x7f8264144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- 172.21.15.12:0/509021464 >> 172.21.15.12:0/509021464 conn(0x7f82640b93b0 msgr2=0x7f82640b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- 172.21.15.12:0/509021464 shutdown_connections 2024-04-27T07:04:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- 172.21.15.12:0/509021464 wait complete. 2024-04-27T07:04:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 Processor -- start 2024-04-27T07:04:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- start start 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826409d440 0x7f82640af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82640b69a0 0x7f82641e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 0x7f82641de550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f82640b9ea0 con 0x7f8264141670 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8264144ea0 con 0x7f82640b69a0 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f82640ba020 con 0x7f826409d440 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8269c52640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 0x7f82641de550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8269c52640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 0x7f82641de550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54936/0 (socket says 172.21.15.12:54936) 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8269c52640 1 -- 172.21.15.12:0/1667039111 learned_addr learned my addr 172.21.15.12:0/1667039111 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8268c50640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82640b69a0 0x7f82641e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8269451640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826409d440 0x7f82640af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8269c52640 1 -- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826409d440 msgr2=0x7f82640af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8269c52640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826409d440 0x7f82640af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8269c52640 1 -- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82640b69a0 msgr2=0x7f82641e1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8269c52640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82640b69a0 0x7f82641e1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8269c52640 1 -- 172.21.15.12:0/1667039111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8264140240 con 0x7f8264141670 2024-04-27T07:04:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8268c50640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82640b69a0 0x7f82641e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8269c52640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 0x7f82641de550 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f825c00c430 tx=0x7f825c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8259ffb640 1 -- 172.21.15.12:0/1667039111 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c005220 con 0x7f8264141670 2024-04-27T07:04:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8259ffb640 1 -- 172.21.15.12:0/1667039111 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825c0053c0 con 0x7f8264141670 2024-04-27T07:04:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f8259ffb640 1 -- 172.21.15.12:0/1667039111 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c007270 con 0x7f8264141670 2024-04-27T07:04:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- 172.21.15.12:0/1667039111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826413c130 con 0x7f8264141670 2024-04-27T07:04:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.957+0000 7f826b6b5640 1 -- 172.21.15.12:0/1667039111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826413c3a0 con 0x7f8264141670 2024-04-27T07:04:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.961+0000 7f826b6b5640 1 -- 172.21.15.12:0/1667039111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8264068c20 con 0x7f8264141670 2024-04-27T07:04:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.961+0000 7f8259ffb640 1 -- 172.21.15.12:0/1667039111 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f825c019020 con 0x7f8264141670 2024-04-27T07:04:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.961+0000 7f8259ffb640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8248068380 0x7f824806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.961+0000 7f8259ffb640 1 -- 172.21.15.12:0/1667039111 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f825c085d10 con 0x7f8264141670 2024-04-27T07:04:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.961+0000 7f8269451640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8248068380 0x7f824806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.961+0000 7f8269451640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8248068380 0x7f824806a840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8264143640 tx=0x7f826000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:42.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:42.965+0000 7f8259ffb640 1 -- 172.21.15.12:0/1667039111 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f825c056360 con 0x7f8264141670 2024-04-27T07:04:43.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.125+0000 7f826b6b5640 1 -- 172.21.15.12:0/1667039111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f826406cc30 con 0x7f8264141670 2024-04-27T07:04:43.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.125+0000 7f8259ffb640 1 -- 172.21.15.12:0/1667039111 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f825c055c60 con 0x7f8264141670 2024-04-27T07:04:43.131 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:04:43.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 -- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8248068380 msgr2=0x7f824806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8248068380 0x7f824806a840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8264143640 tx=0x7f826000b040 comp rx=0 tx=0).stop 2024-04-27T07:04:43.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 -- 172.21.15.12:0/1667039111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 msgr2=0x7f82641de550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 0x7f82641de550 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f825c00c430 tx=0x7f825c00c8f0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 -- 172.21.15.12:0/1667039111 shutdown_connections 2024-04-27T07:04:43.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8248068380 0x7f824806a840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8264141670 0x7f82641de550 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82640b69a0 0x7f82641e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 --2- 172.21.15.12:0/1667039111 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f826409d440 0x7f82640af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 -- 172.21.15.12:0/1667039111 >> 172.21.15.12:0/1667039111 conn(0x7f82640b93b0 msgr2=0x7f826409b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 -- 172.21.15.12:0/1667039111 shutdown_connections 2024-04-27T07:04:43.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.129+0000 7f826b6b5640 1 -- 172.21.15.12:0/1667039111 wait complete. 2024-04-27T07:04:43.150 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-04-27T07:04:43.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c87cd640 1 Processor -- start 2024-04-27T07:04:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c87cd640 1 -- start start 2024-04-27T07:04:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c87cd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70c00b9ae0 0x7f70c0073c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c87cd640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f70c01480a0 0x7f70c0148500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c87cd640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c013e5b0 0x7f70c013ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c87cd640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f70c0148a40 con 0x7f70c00b9ae0 2024-04-27T07:04:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c87cd640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f70c00b9ec0 con 0x7f70c01480a0 2024-04-27T07:04:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c87cd640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f70c009ddb0 con 0x7f70c013e5b0 2024-04-27T07:04:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6569640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70c00b9ae0 0x7f70c0073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6569640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70c00b9ae0 0x7f70c0073c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54950/0 (socket says 172.21.15.12:54950) 2024-04-27T07:04:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6569640 1 -- 172.21.15.12:0/4278829834 learned_addr learned my addr 172.21.15.12:0/4278829834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c5d68640 1 --2- 172.21.15.12:0/4278829834 >> v2:172.21.15.53:3300/0 conn(0x7f70c01480a0 0x7f70c0148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6d6a640 1 --2- 172.21.15.12:0/4278829834 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c013e5b0 0x7f70c013ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6569640 1 -- 172.21.15.12:0/4278829834 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c013e5b0 msgr2=0x7f70c013ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6569640 1 --2- 172.21.15.12:0/4278829834 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c013e5b0 0x7f70c013ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6569640 1 -- 172.21.15.12:0/4278829834 >> v2:172.21.15.53:3300/0 conn(0x7f70c01480a0 msgr2=0x7f70c0148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6569640 1 --2- 172.21.15.12:0/4278829834 >> v2:172.21.15.53:3300/0 conn(0x7f70c01480a0 0x7f70c0148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6569640 1 -- 172.21.15.12:0/4278829834 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c009e090 con 0x7f70c00b9ae0 2024-04-27T07:04:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c5d68640 1 --2- 172.21.15.12:0/4278829834 >> v2:172.21.15.53:3300/0 conn(0x7f70c01480a0 0x7f70c0148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:43.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6d6a640 1 --2- 172.21.15.12:0/4278829834 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c013e5b0 0x7f70c013ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.273+0000 7f70c6569640 1 --2- 172.21.15.12:0/4278829834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c00b9ae0 0x7f70c0073c60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f70c006f030 tx=0x7f70bc01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e085fbc2f6930629 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5567640 1 -- 172.21.15.12:0/4278829834 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00b2d0 con 0x7f70c00b9ae0 2024-04-27T07:04:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5567640 1 -- 172.21.15.12:0/4278829834 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70bc00b470 con 0x7f70c00b9ae0 2024-04-27T07:04:43.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5567640 1 -- 172.21.15.12:0/4278829834 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0a7aa0 con 0x7f70c00b9ae0 2024-04-27T07:04:43.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 -- 172.21.15.12:0/4278829834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c00b9ae0 msgr2=0x7f70c0073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 --2- 172.21.15.12:0/4278829834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c00b9ae0 0x7f70c0073c60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f70c006f030 tx=0x7f70bc01ebf0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 -- 172.21.15.12:0/4278829834 shutdown_connections 2024-04-27T07:04:43.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 --2- 172.21.15.12:0/4278829834 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c013e5b0 0x7f70c013ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 --2- 172.21.15.12:0/4278829834 >> v2:172.21.15.53:3300/0 conn(0x7f70c01480a0 0x7f70c0148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 --2- 172.21.15.12:0/4278829834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c00b9ae0 0x7f70c0073c60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 -- 172.21.15.12:0/4278829834 >> 172.21.15.12:0/4278829834 conn(0x7f70c0067d10 msgr2=0x7f70c0068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:43.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 -- 172.21.15.12:0/4278829834 shutdown_connections 2024-04-27T07:04:43.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 -- 172.21.15.12:0/4278829834 wait complete. 2024-04-27T07:04:43.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 Processor -- start 2024-04-27T07:04:43.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 -- start start 2024-04-27T07:04:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c00b9ae0 0x7f70c0139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c013e5b0 0x7f70c00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c01480a0 0x7f70c014d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f70c00b45c0 con 0x7f70c013e5b0 2024-04-27T07:04:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f70c009db90 con 0x7f70c01480a0 2024-04-27T07:04:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c87cd640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f70c00b4740 con 0x7f70c00b9ae0 2024-04-27T07:04:43.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5d68640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c013e5b0 0x7f70c00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5d68640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c013e5b0 0x7f70c00b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54966/0 (socket says 172.21.15.12:54966) 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5d68640 1 -- 172.21.15.12:0/2679212172 learned_addr learned my addr 172.21.15.12:0/2679212172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c6569640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c00b9ae0 0x7f70c0139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c6d6a640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c01480a0 0x7f70c014d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5d68640 1 -- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c00b9ae0 msgr2=0x7f70c0139060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5d68640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c00b9ae0 0x7f70c0139060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5d68640 1 -- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c01480a0 msgr2=0x7f70c014d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5d68640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c01480a0 0x7f70c014d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5d68640 1 -- 172.21.15.12:0/2679212172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c00b0c40 con 0x7f70c013e5b0 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c6d6a640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c01480a0 0x7f70c014d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70c5d68640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c013e5b0 0x7f70c00b1b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f70b000c7f0 tx=0x7f70b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.277+0000 7f70b6ffd640 1 -- 172.21.15.12:0/2679212172 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0015070 con 0x7f70c013e5b0 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.281+0000 7f70b6ffd640 1 -- 172.21.15.12:0/2679212172 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b00077c0 con 0x7f70c013e5b0 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.281+0000 7f70c87cd640 1 -- 172.21.15.12:0/2679212172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c00b3330 con 0x7f70c013e5b0 2024-04-27T07:04:43.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.281+0000 7f70b6ffd640 1 -- 172.21.15.12:0/2679212172 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0007bb0 con 0x7f70c013e5b0 2024-04-27T07:04:43.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.281+0000 7f70c87cd640 1 -- 172.21.15.12:0/2679212172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c00b3050 con 0x7f70c013e5b0 2024-04-27T07:04:43.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.281+0000 7f70c87cd640 1 -- 172.21.15.12:0/2679212172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c0068c20 con 0x7f70c013e5b0 2024-04-27T07:04:43.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.281+0000 7f70b6ffd640 1 -- 172.21.15.12:0/2679212172 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f70b0018070 con 0x7f70c013e5b0 2024-04-27T07:04:43.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.281+0000 7f70b6ffd640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70a4068380 0x7f70a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.281+0000 7f70b6ffd640 1 -- 172.21.15.12:0/2679212172 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f70b00867f0 con 0x7f70c013e5b0 2024-04-27T07:04:43.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.281+0000 7f70c6569640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70a4068380 0x7f70a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.281+0000 7f70c6569640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70a4068380 0x7f70a406a840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0b0000 tx=0x7f70bc0a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:43.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.285+0000 7f70b6ffd640 1 -- 172.21.15.12:0/2679212172 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f70b0056da0 con 0x7f70c013e5b0 2024-04-27T07:04:43.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.509+0000 7f70c87cd640 1 -- 172.21.15.12:0/2679212172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7f70c0142320 con 0x7f70c013e5b0 2024-04-27T07:04:43.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70b6ffd640 1 -- 172.21.15.12:0/2679212172 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v45) ==== 137+0+0 (secure 0 0 0) 0x7f70b00566a0 con 0x7f70c013e5b0 2024-04-27T07:04:43.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 -- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70a4068380 msgr2=0x7f70a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70a4068380 0x7f70a406a840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0b0000 tx=0x7f70bc0a9070 comp rx=0 tx=0).stop 2024-04-27T07:04:43.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 -- 172.21.15.12:0/2679212172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c013e5b0 msgr2=0x7f70c00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c013e5b0 0x7f70c00b1b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f70b000c7f0 tx=0x7f70b000ccb0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 -- 172.21.15.12:0/2679212172 shutdown_connections 2024-04-27T07:04:43.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f70a4068380 0x7f70a406a840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f70c01480a0 0x7f70c014d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70c013e5b0 0x7f70c00b1b40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 --2- 172.21.15.12:0/2679212172 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f70c00b9ae0 0x7f70c0139060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 -- 172.21.15.12:0/2679212172 >> 172.21.15.12:0/2679212172 conn(0x7f70c0067d10 msgr2=0x7f70c00ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:43.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 -- 172.21.15.12:0/2679212172 shutdown_connections 2024-04-27T07:04:43.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.741+0000 7f70c87cd640 1 -- 172.21.15.12:0/2679212172 wait complete. 2024-04-27T07:04:43.769 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-27T07:04:43.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209f338640 1 Processor -- start 2024-04-27T07:04:43.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209f338640 1 -- start start 2024-04-27T07:04:43.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209f338640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20980b1360 0x7f2098153650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209f338640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2098149210 con 0x7f20980af7a0 2024-04-27T07:04:43.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209f338640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2098147da0 con 0x7f209806bde0 2024-04-27T07:04:43.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209f338640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f20981480f0 con 0x7f20980b1360 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209c8d3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f20980af7a0 0x7f209814a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43170/0 (socket says 172.21.15.12:43170) 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209c8d3640 1 -- 172.21.15.12:0/2591926709 learned_addr learned my addr 172.21.15.12:0/2591926709 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209d8d5640 1 --2- 172.21.15.12:0/2591926709 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20980b1360 0x7f2098153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4162925046 0 0) 0x7f2098149210 con 0x7f20980af7a0 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2084003530 con 0x7f20980af7a0 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2853587611 0 0) 0x7f2084003530 con 0x7f20980af7a0 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20980b1360 msgr2=0x7f2098153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 --2- 172.21.15.12:0/2591926709 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20980b1360 0x7f2098153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 >> v1:172.21.15.53:6789/0 conn(0x7f209806bde0 legacy=0x7f2098148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20980afc30 con 0x7f20980af7a0 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f208c0016a0 con 0x7f20980af7a0 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 >> v1:172.21.15.12:6789/0 conn(0x7f20980af7a0 legacy=0x7f209814a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 --2- 172.21.15.12:0/2591926709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20840039f0 0x7f2084003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 --2- 172.21.15.12:0/2591926709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2084004810 0x7f2084004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 --2- 172.21.15.12:0/2591926709 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20980b1360 0x7f2084005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f20981480f0 con 0x7f20840039f0 2024-04-27T07:04:43.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f208400bb30 con 0x7f2084004810 2024-04-27T07:04:43.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f208400bd30 con 0x7f20980b1360 2024-04-27T07:04:43.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209d0d4640 1 --2- 172.21.15.12:0/2591926709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20840039f0 0x7f2084003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209d0d4640 1 -- 172.21.15.12:0/2591926709 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20980b1360 msgr2=0x7f2084005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:43.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209d0d4640 1 --2- 172.21.15.12:0/2591926709 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20980b1360 0x7f2084005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209d0d4640 1 -- 172.21.15.12:0/2591926709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2084004810 msgr2=0x7f2084004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:43.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209d0d4640 1 --2- 172.21.15.12:0/2591926709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2084004810 0x7f2084004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.892+0000 7f209d0d4640 1 -- 172.21.15.12:0/2591926709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2084003530 con 0x7f20840039f0 2024-04-27T07:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209d0d4640 1 --2- 172.21.15.12:0/2591926709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20840039f0 0x7f2084003e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2094001e70 tx=0x7f209401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e921ccabb37baa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20940af070 con 0x7f20840039f0 2024-04-27T07:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2094003b80 con 0x7f20840039f0 2024-04-27T07:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f208bfff640 1 -- 172.21.15.12:0/2591926709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094001a90 con 0x7f20840039f0 2024-04-27T07:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 -- 172.21.15.12:0/2591926709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20840039f0 msgr2=0x7f2084003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 --2- 172.21.15.12:0/2591926709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20840039f0 0x7f2084003e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2094001e70 tx=0x7f209401c780 comp rx=0 tx=0).stop 2024-04-27T07:04:43.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209d8d5640 1 -- 172.21.15.12:0/2591926709 reap_dead start 2024-04-27T07:04:43.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 -- 172.21.15.12:0/2591926709 shutdown_connections 2024-04-27T07:04:43.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 -- 172.21.15.12:0/2591926709 >> 172.21.15.12:0/2591926709 conn(0x7f20980b7060 msgr2=0x7f20980b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:43.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 -- 172.21.15.12:0/2591926709 shutdown_connections 2024-04-27T07:04:43.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 -- 172.21.15.12:0/2591926709 wait complete. 2024-04-27T07:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 Processor -- start 2024-04-27T07:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 -- start start 2024-04-27T07:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20840039f0 0x7f2098073310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980b28d0 0x7f20981e0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20981e07b0 0x7f20981e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f209814b120 con 0x7f20981e07b0 2024-04-27T07:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f209814afa0 con 0x7f20980b28d0 2024-04-27T07:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209f338640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f2098153e60 con 0x7f20840039f0 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209d8d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20981e07b0 0x7f20981e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209d8d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20981e07b0 0x7f20981e37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54984/0 (socket says 172.21.15.12:54984) 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.896+0000 7f209d8d5640 1 -- 172.21.15.12:0/3325693010 learned_addr learned my addr 172.21.15.12:0/3325693010 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209c8d3640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980b28d0 0x7f20981e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209d0d4640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20840039f0 0x7f2098073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209d8d5640 1 -- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20840039f0 msgr2=0x7f2098073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209d8d5640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20840039f0 0x7f2098073310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209d8d5640 1 -- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980b28d0 msgr2=0x7f20981e0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209d8d5640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980b28d0 0x7f20981e0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209d8d5640 1 -- 172.21.15.12:0/3325693010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20980b2de0 con 0x7f20981e07b0 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209d8d5640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20981e07b0 0x7f20981e37e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f20980b7010 tx=0x7f209000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f2089ffb640 1 -- 172.21.15.12:0/3325693010 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090017070 con 0x7f20981e07b0 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f2089ffb640 1 -- 172.21.15.12:0/3325693010 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f209000c680 con 0x7f20981e07b0 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209f338640 1 -- 172.21.15.12:0/3325693010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2098071700 con 0x7f20981e07b0 2024-04-27T07:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f2089ffb640 1 -- 172.21.15.12:0/3325693010 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209000ca50 con 0x7f20981e07b0 2024-04-27T07:04:43.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209f338640 1 -- 172.21.15.12:0/3325693010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20980a0660 con 0x7f20981e07b0 2024-04-27T07:04:43.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209f338640 1 -- 172.21.15.12:0/3325693010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2098068c20 con 0x7f20981e07b0 2024-04-27T07:04:43.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f2089ffb640 1 -- 172.21.15.12:0/3325693010 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f209001a070 con 0x7f20981e07b0 2024-04-27T07:04:43.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f2089ffb640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f207c068380 0x7f207c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:43.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f2089ffb640 1 -- 172.21.15.12:0/3325693010 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2090086710 con 0x7f20981e07b0 2024-04-27T07:04:43.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209d0d4640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f207c068380 0x7f207c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:43.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.900+0000 7f209d0d4640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f207c068380 0x7f207c06a840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f20940ac000 tx=0x7f2094001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:43.904+0000 7f2089ffb640 1 -- 172.21.15.12:0/3325693010 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2090056d60 con 0x7f20981e07b0 2024-04-27T07:04:44.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.068+0000 7f209f338640 1 -- 172.21.15.12:0/3325693010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f20980b55c0 con 0x7f20981e07b0 2024-04-27T07:04:44.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f2089ffb640 1 -- 172.21.15.12:0/3325693010 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f2090056660 con 0x7f20981e07b0 2024-04-27T07:04:44.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 -- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f207c068380 msgr2=0x7f207c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f207c068380 0x7f207c06a840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f20940ac000 tx=0x7f2094001520 comp rx=0 tx=0).stop 2024-04-27T07:04:44.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 -- 172.21.15.12:0/3325693010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20981e07b0 msgr2=0x7f20981e37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20981e07b0 0x7f20981e37e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f20980b7010 tx=0x7f209000ec00 comp rx=0 tx=0).stop 2024-04-27T07:04:44.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 -- 172.21.15.12:0/3325693010 shutdown_connections 2024-04-27T07:04:44.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f207c068380 0x7f207c06a840 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20981e07b0 0x7f20981e37e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20980b28d0 0x7f20981e0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 --2- 172.21.15.12:0/3325693010 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20840039f0 0x7f2098073310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 -- 172.21.15.12:0/3325693010 >> 172.21.15.12:0/3325693010 conn(0x7f20980b7060 msgr2=0x7f209814a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:44.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 -- 172.21.15.12:0/3325693010 shutdown_connections 2024-04-27T07:04:44.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.076+0000 7f209f338640 1 -- 172.21.15.12:0/3325693010 wait complete. 2024-04-27T07:04:44.100 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-27T07:04:44.100 DEBUG:teuthology.orchestra.run.smithi012:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-27T07:04:44.154 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-27T07:04:44.220 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-27T07:04:44.220 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:04:44.220 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:04:44.226 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:04:44.226 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:04:44.273 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:44.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20ae7fe640 1 Processor -- start 2024-04-27T07:04:44.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20ae7fe640 1 -- start start 2024-04-27T07:04:44.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20ae7fe640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b64c0 0x7f20a80b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20ae7fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f20a80b5ae0 0x7f20a8148810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20ae7fe640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a80b7560 con 0x7f20a8141ad0 2024-04-27T07:04:44.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20ae7fe640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f20a80b77d0 con 0x7f20a80b5ae0 2024-04-27T07:04:44.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20ae7fe640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f20a80b68a0 con 0x7f20a80b64c0 2024-04-27T07:04:44.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20a7fff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b64c0 0x7f20a80b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20a77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f20a80b5ae0 0x7f20a8148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20a77fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f20a80b5ae0 0x7f20a8148810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:40034/0 (socket says 172.21.15.12:40034) 2024-04-27T07:04:44.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20a77fe640 1 -- 172.21.15.12:0/637761644 learned_addr learned my addr 172.21.15.12:0/637761644 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:44.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20a7fff640 1 -- 172.21.15.12:0/637761644 >> v2:172.21.15.53:3300/0 conn(0x7f20a80b5ae0 msgr2=0x7f20a8148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:44.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20a7fff640 1 --2- 172.21.15.12:0/637761644 >> v2:172.21.15.53:3300/0 conn(0x7f20a80b5ae0 0x7f20a8148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20a7fff640 1 -- 172.21.15.12:0/637761644 >> v1:172.21.15.53:6789/0 conn(0x7f20a8141ad0 legacy=0x7f20a814b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.436+0000 7f20a7fff640 1 -- 172.21.15.12:0/637761644 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a8148010 con 0x7f20a80b64c0 2024-04-27T07:04:44.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20a7fff640 1 --2- 172.21.15.12:0/637761644 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b64c0 0x7f20a80b7f60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20980097c0 tx=0x7f209801c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=589656ca09dfd5db server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:44.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20a6ffd640 1 -- 172.21.15.12:0/637761644 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20980b0070 con 0x7f20a80b64c0 2024-04-27T07:04:44.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20a6ffd640 1 -- 172.21.15.12:0/637761644 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20980ab600 con 0x7f20a80b64c0 2024-04-27T07:04:44.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20a6ffd640 1 -- 172.21.15.12:0/637761644 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20980ab9d0 con 0x7f20a80b64c0 2024-04-27T07:04:44.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20ae7fe640 1 -- 172.21.15.12:0/637761644 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b64c0 msgr2=0x7f20a80b7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20ae7fe640 1 --2- 172.21.15.12:0/637761644 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b64c0 0x7f20a80b7f60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f20980097c0 tx=0x7f209801c350 comp rx=0 tx=0).stop 2024-04-27T07:04:44.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20ae7fe640 1 -- 172.21.15.12:0/637761644 shutdown_connections 2024-04-27T07:04:44.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20ae7fe640 1 --2- 172.21.15.12:0/637761644 >> v2:172.21.15.53:3300/0 conn(0x7f20a80b5ae0 0x7f20a8148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20ae7fe640 1 --2- 172.21.15.12:0/637761644 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b64c0 0x7f20a80b7f60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20ae7fe640 1 -- 172.21.15.12:0/637761644 >> 172.21.15.12:0/637761644 conn(0x7f20a8073c00 msgr2=0x7f20a80bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:44.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20ae7fe640 1 -- 172.21.15.12:0/637761644 shutdown_connections 2024-04-27T07:04:44.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20ae7fe640 1 -- 172.21.15.12:0/637761644 wait complete. 2024-04-27T07:04:44.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20ae7fe640 1 Processor -- start 2024-04-27T07:04:44.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.440+0000 7f20ae7fe640 1 -- start start 2024-04-27T07:04:44.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20ae7fe640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b5ae0 0x7f20a81444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20ae7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a80b64c0 0x7f20a81450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20ae7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a8141ad0 0x7f20a81e07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20ae7fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f20a80b51c0 con 0x7f20a80b64c0 2024-04-27T07:04:44.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20ae7fe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20a80b5040 con 0x7f20a8141ad0 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20ae7fe640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f20a8155660 con 0x7f20a80b5ae0 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a80b64c0 0x7f20a81450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a80b64c0 0x7f20a81450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54994/0 (socket says 172.21.15.12:54994) 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a77fe640 1 -- 172.21.15.12:0/1876151182 learned_addr learned my addr 172.21.15.12:0/1876151182 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a7fff640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b5ae0 0x7f20a81444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20acd9b640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a8141ad0 0x7f20a81e07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a77fe640 1 -- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b5ae0 msgr2=0x7f20a81444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a77fe640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b5ae0 0x7f20a81444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a77fe640 1 -- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a8141ad0 msgr2=0x7f20a81e07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a77fe640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a8141ad0 0x7f20a81e07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a77fe640 1 -- 172.21.15.12:0/1876151182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a8146390 con 0x7f20a80b64c0 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a77fe640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a80b64c0 0x7f20a81450e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f209c00e820 tx=0x7f209c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a4ff9640 1 -- 172.21.15.12:0/1876151182 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c017070 con 0x7f20a80b64c0 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a4ff9640 1 -- 172.21.15.12:0/1876151182 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f209c00c940 con 0x7f20a80b64c0 2024-04-27T07:04:44.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a4ff9640 1 -- 172.21.15.12:0/1876151182 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c00cd30 con 0x7f20a80b64c0 2024-04-27T07:04:44.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20ae7fe640 1 -- 172.21.15.12:0/1876151182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a8143ad0 con 0x7f20a80b64c0 2024-04-27T07:04:44.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20ae7fe640 1 -- 172.21.15.12:0/1876151182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a81432d0 con 0x7f20a80b64c0 2024-04-27T07:04:44.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20ae7fe640 1 -- 172.21.15.12:0/1876151182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a8068c20 con 0x7f20a80b64c0 2024-04-27T07:04:44.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a4ff9640 1 -- 172.21.15.12:0/1876151182 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f209c01a070 con 0x7f20a80b64c0 2024-04-27T07:04:44.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a4ff9640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2094068380 0x7f209406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a4ff9640 1 -- 172.21.15.12:0/1876151182 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f209c086850 con 0x7f20a80b64c0 2024-04-27T07:04:44.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.444+0000 7f20a7fff640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2094068380 0x7f209406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.448+0000 7f20a7fff640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2094068380 0x7f209406a840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f20980ad000 tx=0x7f20980a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:44.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.448+0000 7f20a4ff9640 1 -- 172.21.15.12:0/1876151182 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f209c056e20 con 0x7f20a80b64c0 2024-04-27T07:04:44.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.612+0000 7f20ae7fe640 1 -- 172.21.15.12:0/1876151182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20a80b53e0 con 0x7f20a80b64c0 2024-04-27T07:04:44.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20a4ff9640 1 -- 172.21.15.12:0/1876151182 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f209c056720 con 0x7f20a80b64c0 2024-04-27T07:04:44.621 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:44.621 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:44.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 -- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2094068380 msgr2=0x7f209406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2094068380 0x7f209406a840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f20980ad000 tx=0x7f20980a6000 comp rx=0 tx=0).stop 2024-04-27T07:04:44.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 -- 172.21.15.12:0/1876151182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a80b64c0 msgr2=0x7f20a81450e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a80b64c0 0x7f20a81450e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f209c00e820 tx=0x7f209c00ece0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 -- 172.21.15.12:0/1876151182 shutdown_connections 2024-04-27T07:04:44.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f2094068380 0x7f209406a840 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a8141ad0 0x7f20a81e07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20a80b64c0 0x7f20a81450e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 --2- 172.21.15.12:0/1876151182 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f20a80b5ae0 0x7f20a81444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 -- 172.21.15.12:0/1876151182 >> 172.21.15.12:0/1876151182 conn(0x7f20a8073c00 msgr2=0x7f20a80ba600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:44.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 -- 172.21.15.12:0/1876151182 shutdown_connections 2024-04-27T07:04:44.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.616+0000 7f20ae7fe640 1 -- 172.21.15.12:0/1876151182 wait complete. 2024-04-27T07:04:44.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:12.742740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:13.912898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:44.641 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0957803640 1 Processor -- start 2024-04-27T07:04:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0957803640 1 -- start start 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0957803640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0950149440 0x7f09500b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0957803640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f09500b84d0 0x7f0950150570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0957803640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0950149d20 con 0x7f09500b8e40 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0957803640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f095014a000 con 0x7f09500b84d0 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0957803640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f09501401d0 con 0x7f0950149440 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0955da0640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f09500b84d0 0x7f0950150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0955da0640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f09500b84d0 0x7f0950150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54996/0 (socket says 172.21.15.12:54996) 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0955da0640 1 -- 172.21.15.12:0/2910563203 learned_addr learned my addr 172.21.15.12:0/2910563203 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f095559f640 1 --2- 172.21.15.12:0/2910563203 >> v2:172.21.15.53:3300/0 conn(0x7f0950149440 0x7f09500b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0955da0640 1 -- 172.21.15.12:0/2910563203 >> v2:172.21.15.53:3300/0 conn(0x7f0950149440 msgr2=0x7f09500b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0955da0640 1 --2- 172.21.15.12:0/2910563203 >> v2:172.21.15.53:3300/0 conn(0x7f0950149440 0x7f09500b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0955da0640 1 -- 172.21.15.12:0/2910563203 >> v1:172.21.15.53:6789/0 conn(0x7f09500b8e40 legacy=0x7f095006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:44.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.768+0000 7f0955da0640 1 -- 172.21.15.12:0/2910563203 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0950149820 con 0x7f09500b84d0 2024-04-27T07:04:44.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0955da0640 1 --2- 172.21.15.12:0/2910563203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09500b84d0 0x7f0950150570 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f09500710f0 tx=0x7f094c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf29d289a8d751d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:44.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0947fff640 1 -- 172.21.15.12:0/2910563203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0b0070 con 0x7f09500b84d0 2024-04-27T07:04:44.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0947fff640 1 -- 172.21.15.12:0/2910563203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094c0ab600 con 0x7f09500b84d0 2024-04-27T07:04:44.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0947fff640 1 -- 172.21.15.12:0/2910563203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0ab9d0 con 0x7f09500b84d0 2024-04-27T07:04:44.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0957803640 1 -- 172.21.15.12:0/2910563203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09500b84d0 msgr2=0x7f0950150570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0957803640 1 --2- 172.21.15.12:0/2910563203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09500b84d0 0x7f0950150570 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f09500710f0 tx=0x7f094c01c450 comp rx=0 tx=0).stop 2024-04-27T07:04:44.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0957803640 1 -- 172.21.15.12:0/2910563203 shutdown_connections 2024-04-27T07:04:44.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0957803640 1 --2- 172.21.15.12:0/2910563203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09500b84d0 0x7f0950150570 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0957803640 1 --2- 172.21.15.12:0/2910563203 >> v2:172.21.15.53:3300/0 conn(0x7f0950149440 0x7f09500b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0957803640 1 -- 172.21.15.12:0/2910563203 >> 172.21.15.12:0/2910563203 conn(0x7f095009c480 msgr2=0x7f095009c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:44.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0957803640 1 -- 172.21.15.12:0/2910563203 shutdown_connections 2024-04-27T07:04:44.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0957803640 1 -- 172.21.15.12:0/2910563203 wait complete. 2024-04-27T07:04:44.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0957803640 1 Processor -- start 2024-04-27T07:04:44.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.772+0000 7f0957803640 1 -- start start 2024-04-27T07:04:44.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0957803640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09500b84d0 0x7f095009eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0957803640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09500b8e40 0x7f095013cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0957803640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0950149440 0x7f095013d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0957803640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0950153660 con 0x7f0950149440 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0957803640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f09501534e0 con 0x7f09500b84d0 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0957803640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f09501537e0 con 0x7f09500b8e40 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0955da0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0950149440 0x7f095013d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0955da0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0950149440 0x7f095013d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55006/0 (socket says 172.21.15.12:55006) 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0955da0640 1 -- 172.21.15.12:0/3231977795 learned_addr learned my addr 172.21.15.12:0/3231977795 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f095559f640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09500b84d0 0x7f095009eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0954d9e640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09500b8e40 0x7f095013cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0955da0640 1 -- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09500b8e40 msgr2=0x7f095013cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0955da0640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09500b8e40 0x7f095013cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0955da0640 1 -- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09500b84d0 msgr2=0x7f095009eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0955da0640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09500b84d0 0x7f095009eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0955da0640 1 -- 172.21.15.12:0/3231977795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09500b1c00 con 0x7f0950149440 2024-04-27T07:04:44.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f095559f640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09500b84d0 0x7f095009eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:44.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0954d9e640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09500b8e40 0x7f095013cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:44.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0955da0640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0950149440 0x7f095013d320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f094c01a080 tx=0x7f094c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:44.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0945ffb640 1 -- 172.21.15.12:0/3231977795 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0b8020 con 0x7f0950149440 2024-04-27T07:04:44.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0945ffb640 1 -- 172.21.15.12:0/3231977795 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094c0b46d0 con 0x7f0950149440 2024-04-27T07:04:44.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0945ffb640 1 -- 172.21.15.12:0/3231977795 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0b0040 con 0x7f0950149440 2024-04-27T07:04:44.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0957803640 1 -- 172.21.15.12:0/3231977795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0950146580 con 0x7f0950149440 2024-04-27T07:04:44.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0957803640 1 -- 172.21.15.12:0/3231977795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095009da50 con 0x7f0950149440 2024-04-27T07:04:44.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0957803640 1 -- 172.21.15.12:0/3231977795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0950069330 con 0x7f0950149440 2024-04-27T07:04:44.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0945ffb640 1 -- 172.21.15.12:0/3231977795 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f094c0bb020 con 0x7f0950149440 2024-04-27T07:04:44.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0945ffb640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0934068380 0x7f093406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:44.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f0945ffb640 1 -- 172.21.15.12:0/3231977795 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f094c120670 con 0x7f0950149440 2024-04-27T07:04:44.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.776+0000 7f095559f640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0934068380 0x7f093406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:44.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.780+0000 7f095559f640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0934068380 0x7f093406a840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f095009d1e0 tx=0x7f094800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:44.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.780+0000 7f0945ffb640 1 -- 172.21.15.12:0/3231977795 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f094c0c2030 con 0x7f0950149440 2024-04-27T07:04:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 -- 172.21.15.12:0/3231977795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f095006ee20 con 0x7f0950149440 2024-04-27T07:04:44.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0945ffb640 1 -- 172.21.15.12:0/3231977795 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f094c0160d0 con 0x7f0950149440 2024-04-27T07:04:44.953 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 -- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0934068380 msgr2=0x7f093406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0934068380 0x7f093406a840 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f095009d1e0 tx=0x7f094800b040 comp rx=0 tx=0).stop 2024-04-27T07:04:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 -- 172.21.15.12:0/3231977795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0950149440 msgr2=0x7f095013d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0950149440 0x7f095013d320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f094c01a080 tx=0x7f094c0191f0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 -- 172.21.15.12:0/3231977795 shutdown_connections 2024-04-27T07:04:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0934068380 0x7f093406a840 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0950149440 0x7f095013d320 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f09500b8e40 0x7f095013cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 --2- 172.21.15.12:0/3231977795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f09500b84d0 0x7f095009eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 -- 172.21.15.12:0/3231977795 >> 172.21.15.12:0/3231977795 conn(0x7f095009c480 msgr2=0x7f095006e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 -- 172.21.15.12:0/3231977795 shutdown_connections 2024-04-27T07:04:44.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:44.948+0000 7f0957803640 1 -- 172.21.15.12:0/3231977795 wait complete. 2024-04-27T07:04:44.973 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:04:45.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6408a99640 1 Processor -- start 2024-04-27T07:04:45.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6408a99640 1 -- start start 2024-04-27T07:04:45.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6408a99640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f64040b73c0 0x7f64040b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6408a99640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f64040b8af0 0x7f6404148210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6408a99640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64040b8f60 con 0x7f6404148750 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6408a99640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f64040ba470 con 0x7f64040b73c0 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6408a99640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f64040b77a0 con 0x7f64040b8af0 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6402d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6404148750 0x7f6404144e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43176/0 (socket says 172.21.15.12:43176) 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6402d9d640 1 -- 172.21.15.12:0/361595582 learned_addr learned my addr 172.21.15.12:0/361595582 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6401d9b640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f64040b8af0 0x7f6404148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640259c640 1 --2- 172.21.15.12:0/361595582 >> v2:172.21.15.53:3300/0 conn(0x7f64040b73c0 0x7f64040b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4117643109 0 0) 0x7f64040b8f60 con 0x7f6404148750 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63ec003530 con 0x7f6404148750 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 134104553 0 0) 0x7f63ec003530 con 0x7f6404148750 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f64040b8af0 msgr2=0x7f6404148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f64040b8af0 0x7f6404148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 >> v2:172.21.15.53:3300/0 conn(0x7f64040b73c0 msgr2=0x7f64040b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 --2- 172.21.15.12:0/361595582 >> v2:172.21.15.53:3300/0 conn(0x7f64040b73c0 0x7f64040b85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64040b5c90 con 0x7f6404148750 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f63fc001a00 con 0x7f6404148750 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 >> v1:172.21.15.12:6789/0 conn(0x7f6404148750 legacy=0x7f6404144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ec0039c0 0x7f63ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f64040b8af0 0x7f63ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ec004d50 0x7f63ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f64040ba470 con 0x7f63ec004d50 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64040b77a0 con 0x7f63ec0039c0 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f63ec00b9d0 con 0x7f64040b8af0 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6402d9d640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ec004d50 0x7f63ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640259c640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ec0039c0 0x7f63ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6401d9b640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f64040b8af0 0x7f63ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6402d9d640 1 -- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f64040b8af0 msgr2=0x7f63ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6402d9d640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f64040b8af0 0x7f63ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6402d9d640 1 -- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ec0039c0 msgr2=0x7f63ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6402d9d640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ec0039c0 0x7f63ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f6402d9d640 1 -- 172.21.15.12:0/361595582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec003530 con 0x7f63ec004d50 2024-04-27T07:04:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.092+0000 7f640259c640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f63ec0039c0 0x7f63ec003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:45.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6402d9d640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ec004d50 0x7f63ec0051d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0013e0 tx=0x7f63fc01fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b275bee4d6bb2171 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:45.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc00a4e0 con 0x7f63ec004d50 2024-04-27T07:04:45.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63fc0ad020 con 0x7f63ec004d50 2024-04-27T07:04:45.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f640159a640 1 -- 172.21.15.12:0/361595582 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0a9740 con 0x7f63ec004d50 2024-04-27T07:04:45.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 -- 172.21.15.12:0/361595582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ec004d50 msgr2=0x7f63ec0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 --2- 172.21.15.12:0/361595582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63ec004d50 0x7f63ec0051d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0013e0 tx=0x7f63fc01fb00 comp rx=0 tx=0).stop 2024-04-27T07:04:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6402d9d640 1 -- 172.21.15.12:0/361595582 reap_dead start 2024-04-27T07:04:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 -- 172.21.15.12:0/361595582 shutdown_connections 2024-04-27T07:04:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 -- 172.21.15.12:0/361595582 >> 172.21.15.12:0/361595582 conn(0x7f64040ba6d0 msgr2=0x7f64040baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 -- 172.21.15.12:0/361595582 shutdown_connections 2024-04-27T07:04:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 -- 172.21.15.12:0/361595582 wait complete. 2024-04-27T07:04:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 Processor -- start 2024-04-27T07:04:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 -- start start 2024-04-27T07:04:45.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f640409e4c0 0x7f64041de350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64041de890 0x7f64041444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6404144a10 0x7f64041470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f64040ba0f0 con 0x7f64041de890 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64040b6250 con 0x7f6404144a10 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6408a99640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f64040b7fd0 con 0x7f640409e4c0 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.096+0000 7f6401d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64041de890 0x7f64041444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6401d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64041de890 0x7f64041444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55024/0 (socket says 172.21.15.12:55024) 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6401d9b640 1 -- 172.21.15.12:0/1043258807 learned_addr learned my addr 172.21.15.12:0/1043258807 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f640259c640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f640409e4c0 0x7f64041de350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6402d9d640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6404144a10 0x7f64041470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6401d9b640 1 -- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f640409e4c0 msgr2=0x7f64041de350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6401d9b640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f640409e4c0 0x7f64041de350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6401d9b640 1 -- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6404144a10 msgr2=0x7f64041470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6401d9b640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6404144a10 0x7f64041470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6401d9b640 1 -- 172.21.15.12:0/1043258807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64040b4590 con 0x7f64041de890 2024-04-27T07:04:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6401d9b640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64041de890 0x7f64041444d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f63f80115e0 tx=0x7f63f8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f63f2ffd640 1 -- 172.21.15.12:0/1043258807 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f800f8f0 con 0x7f64041de890 2024-04-27T07:04:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f63f2ffd640 1 -- 172.21.15.12:0/1043258807 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f800fa90 con 0x7f64041de890 2024-04-27T07:04:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6408a99640 1 -- 172.21.15.12:0/1043258807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64041ded10 con 0x7f64041de890 2024-04-27T07:04:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f63f2ffd640 1 -- 172.21.15.12:0/1043258807 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f800fe40 con 0x7f64041de890 2024-04-27T07:04:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6408a99640 1 -- 172.21.15.12:0/1043258807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64040b2110 con 0x7f64041de890 2024-04-27T07:04:45.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f6408a99640 1 -- 172.21.15.12:0/1043258807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6404069330 con 0x7f64041de890 2024-04-27T07:04:45.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f63f2ffd640 1 -- 172.21.15.12:0/1043258807 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f63f8015460 con 0x7f64041de890 2024-04-27T07:04:45.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f63f2ffd640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f63e4068380 0x7f63e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f63f2ffd640 1 -- 172.21.15.12:0/1043258807 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f63f8087290 con 0x7f64041de890 2024-04-27T07:04:45.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f640259c640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f63e4068380 0x7f63e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.100+0000 7f640259c640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f63e4068380 0x7f63e406a840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f64041d1440 tx=0x7f63f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.104+0000 7f63f2ffd640 1 -- 172.21.15.12:0/1043258807 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f64041ded10 con 0x7f64041de890 2024-04-27T07:04:45.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 -- 172.21.15.12:0/1043258807 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64040b4800 con 0x7f64041de890 2024-04-27T07:04:45.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f63f2ffd640 1 -- 172.21.15.12:0/1043258807 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f63f8019030 con 0x7f64041de890 2024-04-27T07:04:45.274 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:04:45.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 -- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f63e4068380 msgr2=0x7f63e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f63e4068380 0x7f63e406a840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f64041d1440 tx=0x7f63f4008040 comp rx=0 tx=0).stop 2024-04-27T07:04:45.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 -- 172.21.15.12:0/1043258807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64041de890 msgr2=0x7f64041444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64041de890 0x7f64041444d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f63f80115e0 tx=0x7f63f8011aa0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 -- 172.21.15.12:0/1043258807 shutdown_connections 2024-04-27T07:04:45.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f63e4068380 0x7f63e406a840 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6404144a10 0x7f64041470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64041de890 0x7f64041444d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 --2- 172.21.15.12:0/1043258807 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f640409e4c0 0x7f64041de350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.268+0000 7f6408a99640 1 -- 172.21.15.12:0/1043258807 >> 172.21.15.12:0/1043258807 conn(0x7f64040ba6d0 msgr2=0x7f640409cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:45.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.272+0000 7f6408a99640 1 -- 172.21.15.12:0/1043258807 shutdown_connections 2024-04-27T07:04:45.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.272+0000 7f6408a99640 1 -- 172.21.15.12:0/1043258807 wait complete. 2024-04-27T07:04:45.293 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:04:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e5016d640 1 Processor -- start 2024-04-27T07:04:45.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e5016d640 1 -- start start 2024-04-27T07:04:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e5016d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e480b5ae0 0x7f1e48142010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e5016d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e480b6860 con 0x7f1e480b6480 2024-04-27T07:04:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e5016d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e48147ee0 con 0x7f1e481489c0 2024-04-27T07:04:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e5016d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f1e481481c0 con 0x7f1e480b5ae0 2024-04-27T07:04:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4df09640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1e480b6480 0x7f1e48141900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43186/0 (socket says 172.21.15.12:43186) 2024-04-27T07:04:45.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4df09640 1 -- 172.21.15.12:0/3155109688 learned_addr learned my addr 172.21.15.12:0/3155109688 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4e70a640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e480b5ae0 0x7f1e48142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902421457 0 0) 0x7f1e480b6860 con 0x7f1e480b6480 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e34003530 con 0x7f1e480b6480 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3266864133 0 0) 0x7f1e34003530 con 0x7f1e480b6480 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e480b5ae0 msgr2=0x7f1e48142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e480b5ae0 0x7f1e48142010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 >> v1:172.21.15.53:6789/0 conn(0x7f1e481489c0 legacy=0x7f1e48150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e481484a0 con 0x7f1e480b6480 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f1e3c001a00 con 0x7f1e480b6480 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 >> v1:172.21.15.12:6789/0 conn(0x7f1e480b6480 legacy=0x7f1e48141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e480b5ae0 0x7f1e340039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 0x7f1e34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e34004d50 0x7f1e340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481481c0 con 0x7f1e34003f70 2024-04-27T07:04:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3400bb60 con 0x7f1e34004d50 2024-04-27T07:04:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f1e3400bd60 con 0x7f1e480b5ae0 2024-04-27T07:04:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4d708640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 0x7f1e34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4df09640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e480b5ae0 0x7f1e340039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4e70a640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e34004d50 0x7f1e340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4d708640 1 -- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e480b5ae0 msgr2=0x7f1e340039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4d708640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e480b5ae0 0x7f1e340039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4d708640 1 -- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e34004d50 msgr2=0x7f1e340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4d708640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e34004d50 0x7f1e340051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.404+0000 7f1e4d708640 1 -- 172.21.15.12:0/3155109688 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e34003530 con 0x7f1e34003f70 2024-04-27T07:04:45.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e4d708640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 0x7f1e34004810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e44001780 tx=0x7f1e4401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8302ea925ce2d621 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:45.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e440013a0 con 0x7f1e34003f70 2024-04-27T07:04:45.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e440052f0 con 0x7f1e34003f70 2024-04-27T07:04:45.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e4cf07640 1 -- 172.21.15.12:0/3155109688 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44003840 con 0x7f1e34003f70 2024-04-27T07:04:45.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 -- 172.21.15.12:0/3155109688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 msgr2=0x7f1e34004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 --2- 172.21.15.12:0/3155109688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 0x7f1e34004810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e44001780 tx=0x7f1e4401c420 comp rx=0 tx=0).stop 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e4e70a640 1 -- 172.21.15.12:0/3155109688 reap_dead start 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 -- 172.21.15.12:0/3155109688 shutdown_connections 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 -- 172.21.15.12:0/3155109688 >> 172.21.15.12:0/3155109688 conn(0x7f1e480bd620 msgr2=0x7f1e480bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 -- 172.21.15.12:0/3155109688 shutdown_connections 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 -- 172.21.15.12:0/3155109688 wait complete. 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 Processor -- start 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 -- start start 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 0x7f1e4813b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e4814a9f0 0x7f1e4813bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e481e43b0 0x7f1e481e4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48154700 con 0x7f1e34003f70 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48154580 con 0x7f1e481e43b0 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e5016d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f1e48154880 con 0x7f1e4814a9f0 2024-04-27T07:04:45.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e4df09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 0x7f1e4813b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e4df09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 0x7f1e4813b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55048/0 (socket says 172.21.15.12:55048) 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.408+0000 7f1e4df09640 1 -- 172.21.15.12:0/204214521 learned_addr learned my addr 172.21.15.12:0/204214521 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4d708640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e4814a9f0 0x7f1e4813bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4e70a640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e481e43b0 0x7f1e481e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4df09640 1 -- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e4814a9f0 msgr2=0x7f1e4813bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4df09640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e4814a9f0 0x7f1e4813bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4df09640 1 -- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e481e43b0 msgr2=0x7f1e481e4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4df09640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e481e43b0 0x7f1e481e4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4df09640 1 -- 172.21.15.12:0/204214521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e481495f0 con 0x7f1e34003f70 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4e70a640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e481e43b0 0x7f1e481e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4df09640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 0x7f1e4813b880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0079e0 tx=0x7f1e3c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e3a7fc640 1 -- 172.21.15.12:0/204214521 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00ee50 con 0x7f1e34003f70 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e3a7fc640 1 -- 172.21.15.12:0/204214521 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3c018070 con 0x7f1e34003f70 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e5016d640 1 -- 172.21.15.12:0/204214521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e48145330 con 0x7f1e34003f70 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e3a7fc640 1 -- 172.21.15.12:0/204214521 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c013660 con 0x7f1e34003f70 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e5016d640 1 -- 172.21.15.12:0/204214521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e48148e90 con 0x7f1e34003f70 2024-04-27T07:04:45.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e5016d640 1 -- 172.21.15.12:0/204214521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e48068c20 con 0x7f1e34003f70 2024-04-27T07:04:45.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e3a7fc640 1 -- 172.21.15.12:0/204214521 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1e3c013800 con 0x7f1e34003f70 2024-04-27T07:04:45.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e3a7fc640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e2c068550 0x7f1e2c06aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e3a7fc640 1 -- 172.21.15.12:0/204214521 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e3c086f60 con 0x7f1e34003f70 2024-04-27T07:04:45.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4d708640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e2c068550 0x7f1e2c06aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.412+0000 7f1e4d708640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e2c068550 0x7f1e2c06aa10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e440ac000 tx=0x7f1e440011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:45.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.416+0000 7f1e3a7fc640 1 -- 172.21.15.12:0/204214521 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e3c0575b0 con 0x7f1e34003f70 2024-04-27T07:04:45.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.572+0000 7f1e5016d640 1 -- 172.21.15.12:0/204214521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1e48151a10 con 0x7f1e34003f70 2024-04-27T07:04:45.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.572+0000 7f1e3a7fc640 1 -- 172.21.15.12:0/204214521 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1e3c056eb0 con 0x7f1e34003f70 2024-04-27T07:04:45.578 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 -- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e2c068550 msgr2=0x7f1e2c06aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e2c068550 0x7f1e2c06aa10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1e440ac000 tx=0x7f1e440011a0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 -- 172.21.15.12:0/204214521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 msgr2=0x7f1e4813b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 0x7f1e4813b880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0079e0 tx=0x7f1e3c00e530 comp rx=0 tx=0).stop 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 -- 172.21.15.12:0/204214521 shutdown_connections 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1e2c068550 0x7f1e2c06aa10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e481e43b0 0x7f1e481e4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1e4814a9f0 0x7f1e4813bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 --2- 172.21.15.12:0/204214521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e34003f70 0x7f1e4813b880 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 -- 172.21.15.12:0/204214521 >> 172.21.15.12:0/204214521 conn(0x7f1e480bd620 msgr2=0x7f1e4813f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 -- 172.21.15.12:0/204214521 shutdown_connections 2024-04-27T07:04:45.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.576+0000 7f1e5016d640 1 -- 172.21.15.12:0/204214521 wait complete. 2024-04-27T07:04:45.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:04:45.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af9b89640 1 Processor -- start 2024-04-27T07:04:45.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af9b89640 1 -- start start 2024-04-27T07:04:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af9b89640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af414a790 0x7f8af414abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af9b89640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8af41506e0 0x7f8af4152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af9b89640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8af406d000 con 0x7f8af406e790 2024-04-27T07:04:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af9b89640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8af40b2740 con 0x7f8af41506e0 2024-04-27T07:04:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af9b89640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8af40b7d20 con 0x7f8af414a790 2024-04-27T07:04:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af37fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8af406e790 0x7f8af40b5130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43190/0 (socket says 172.21.15.12:43190) 2024-04-27T07:04:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af37fe640 1 -- 172.21.15.12:0/286056571 learned_addr learned my addr 172.21.15.12:0/286056571 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af2ffd640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af414a790 0x7f8af414abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af3fff640 1 --2- 172.21.15.12:0/286056571 >> v2:172.21.15.53:3300/0 conn(0x7f8af41506e0 0x7f8af4152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2303183452 0 0) 0x7f8af406d000 con 0x7f8af406e790 2024-04-27T07:04:45.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8adc003530 con 0x7f8af406e790 2024-04-27T07:04:45.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3050339804 0 0) 0x7f8adc003530 con 0x7f8af406e790 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af414a790 msgr2=0x7f8af414abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af414a790 0x7f8af414abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 >> v2:172.21.15.53:3300/0 conn(0x7f8af41506e0 msgr2=0x7f8af4152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 --2- 172.21.15.12:0/286056571 >> v2:172.21.15.53:3300/0 conn(0x7f8af41506e0 0x7f8af4152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af406eb70 con 0x7f8af406e790 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f8ae4001680 con 0x7f8af406e790 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 >> v1:172.21.15.12:6789/0 conn(0x7f8af406e790 legacy=0x7f8af40b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af414a790 0x7f8adc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 0x7f8adc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8adc004d50 0x7f8adc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8af40b2740 con 0x7f8adc003f90 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8af40b7d20 con 0x7f8adc004d50 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8adc00bb00 con 0x7f8af414a790 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af2ffd640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 0x7f8adc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af37fe640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af414a790 0x7f8adc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af2ffd640 1 -- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af414a790 msgr2=0x7f8adc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af3fff640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8adc004d50 0x7f8adc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af2ffd640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af414a790 0x7f8adc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af2ffd640 1 -- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8adc004d50 msgr2=0x7f8adc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af2ffd640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8adc004d50 0x7f8adc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af2ffd640 1 -- 172.21.15.12:0/286056571 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc003530 con 0x7f8adc003f90 2024-04-27T07:04:45.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.716+0000 7f8af3fff640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8adc004d50 0x7f8adc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:45.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af2ffd640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 0x7f8adc004810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ae0 tx=0x7f8ae801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2324cdc1fdee676d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:45.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80b2070 con 0x7f8adc003f90 2024-04-27T07:04:45.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae80a5610 con 0x7f8adc003f90 2024-04-27T07:04:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af27fc640 1 -- 172.21.15.12:0/286056571 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80096b0 con 0x7f8adc003f90 2024-04-27T07:04:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af9b89640 1 -- 172.21.15.12:0/286056571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 msgr2=0x7f8adc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af9b89640 1 --2- 172.21.15.12:0/286056571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 0x7f8adc004810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009ae0 tx=0x7f8ae801ca40 comp rx=0 tx=0).stop 2024-04-27T07:04:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af3fff640 1 -- 172.21.15.12:0/286056571 reap_dead start 2024-04-27T07:04:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af9b89640 1 -- 172.21.15.12:0/286056571 shutdown_connections 2024-04-27T07:04:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af9b89640 1 -- 172.21.15.12:0/286056571 >> 172.21.15.12:0/286056571 conn(0x7f8af4068210 msgr2=0x7f8af40bb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af9b89640 1 -- 172.21.15.12:0/286056571 shutdown_connections 2024-04-27T07:04:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af9b89640 1 -- 172.21.15.12:0/286056571 wait complete. 2024-04-27T07:04:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af9b89640 1 Processor -- start 2024-04-27T07:04:45.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.720+0000 7f8af9b89640 1 -- start start 2024-04-27T07:04:45.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af9b89640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 0x7f8af4142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af9b89640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af4152670 0x7f8af41e3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af9b89640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af41e4260 0x7f8af413e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af9b89640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8af406d9d0 con 0x7f8adc003f90 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af9b89640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8af406d670 con 0x7f8af4152670 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af9b89640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8af40b5840 con 0x7f8af41e4260 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 0x7f8af4142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 0x7f8af4142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55070/0 (socket says 172.21.15.12:55070) 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af37fe640 1 -- 172.21.15.12:0/3690310619 learned_addr learned my addr 172.21.15.12:0/3690310619 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af2ffd640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af4152670 0x7f8af41e3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af3fff640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af41e4260 0x7f8af413e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af37fe640 1 -- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af41e4260 msgr2=0x7f8af413e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af37fe640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af41e4260 0x7f8af413e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af37fe640 1 -- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af4152670 msgr2=0x7f8af41e3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af37fe640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af4152670 0x7f8af41e3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af37fe640 1 -- 172.21.15.12:0/3690310619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af41e46e0 con 0x7f8adc003f90 2024-04-27T07:04:45.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af2ffd640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af4152670 0x7f8af41e3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:45.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af37fe640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 0x7f8af4142de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4000fc0 tx=0x7f8ae400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:45.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8adbfff640 1 -- 172.21.15.12:0/3690310619 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400c6c0 con 0x7f8adc003f90 2024-04-27T07:04:45.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8adbfff640 1 -- 172.21.15.12:0/3690310619 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae4018070 con 0x7f8adc003f90 2024-04-27T07:04:45.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8adbfff640 1 -- 172.21.15.12:0/3690310619 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400cac0 con 0x7f8adc003f90 2024-04-27T07:04:45.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af9b89640 1 -- 172.21.15.12:0/3690310619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af41457c0 con 0x7f8adc003f90 2024-04-27T07:04:45.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af9b89640 1 -- 172.21.15.12:0/3690310619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af413fd00 con 0x7f8adc003f90 2024-04-27T07:04:45.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af9b89640 1 -- 172.21.15.12:0/3690310619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af4069330 con 0x7f8adc003f90 2024-04-27T07:04:45.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8adbfff640 1 -- 172.21.15.12:0/3690310619 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8ae400cc60 con 0x7f8adc003f90 2024-04-27T07:04:45.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8adbfff640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ae00683b0 0x7f8ae006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:45.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8adbfff640 1 -- 172.21.15.12:0/3690310619 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8ae40867b0 con 0x7f8adc003f90 2024-04-27T07:04:45.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.724+0000 7f8af2ffd640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ae00683b0 0x7f8ae006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:45.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.728+0000 7f8af2ffd640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ae00683b0 0x7f8ae006a870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80af000 tx=0x7f8ae80046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:45.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.728+0000 7f8adbfff640 1 -- 172.21.15.12:0/3690310619 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8af41e46e0 con 0x7f8adc003f90 2024-04-27T07:04:45.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.904+0000 7f8af9b89640 1 -- 172.21.15.12:0/3690310619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8af40bb070 con 0x7f8adc003f90 2024-04-27T07:04:45.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.904+0000 7f8adbfff640 1 -- 172.21.15.12:0/3690310619 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f8ae4056cd0 con 0x7f8adc003f90 2024-04-27T07:04:45.910 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2024-04-27T07:04:45.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 -- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ae00683b0 msgr2=0x7f8ae006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ae00683b0 0x7f8ae006a870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80af000 tx=0x7f8ae80046d0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 -- 172.21.15.12:0/3690310619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 msgr2=0x7f8af4142de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 0x7f8af4142de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4000fc0 tx=0x7f8ae400ec10 comp rx=0 tx=0).stop 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 -- 172.21.15.12:0/3690310619 shutdown_connections 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ae00683b0 0x7f8ae006a870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8af41e4260 0x7f8af413e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8af4152670 0x7f8af41e3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 --2- 172.21.15.12:0/3690310619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8adc003f90 0x7f8af4142de0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 -- 172.21.15.12:0/3690310619 >> 172.21.15.12:0/3690310619 conn(0x7f8af4068210 msgr2=0x7f8af40733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 -- 172.21.15.12:0/3690310619 shutdown_connections 2024-04-27T07:04:45.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:45.908+0000 7f8af9b89640 1 -- 172.21.15.12:0/3690310619 wait complete. 2024-04-27T07:04:45.929 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:04:45.930 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:04:45.935 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:46.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.092+0000 7f04c13f8640 1 Processor -- start 2024-04-27T07:04:46.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04c13f8640 1 -- start start 2024-04-27T07:04:46.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04c13f8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc13fef0 0x7f04bc0b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04c13f8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04bc145a00 con 0x7f04bc0b4500 2024-04-27T07:04:46.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04c13f8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04bc0b52b0 con 0x7f04bc145ec0 2024-04-27T07:04:46.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04c13f8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc0b7ae0 con 0x7f04bc13fef0 2024-04-27T07:04:46.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04baffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f04bc0b4500 0x7f04bc13f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43206/0 (socket says 172.21.15.12:43206) 2024-04-27T07:04:46.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04baffd640 1 -- 172.21.15.12:0/2728704278 learned_addr learned my addr 172.21.15.12:0/2728704278 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:46.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04ba7fc640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc13fef0 0x7f04bc0b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 690106738 0 0) 0x7f04ac001120 con 0x7f04bc0b4500 2024-04-27T07:04:46.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04a8003530 con 0x7f04bc0b4500 2024-04-27T07:04:46.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1148373968 0 0) 0x7f04a8003530 con 0x7f04bc0b4500 2024-04-27T07:04:46.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc13fef0 msgr2=0x7f04bc0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:46.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc13fef0 0x7f04bc0b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 >> v1:172.21.15.53:6789/0 conn(0x7f04bc145ec0 legacy=0x7f04bc146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc0b7c60 con 0x7f04bc0b4500 2024-04-27T07:04:46.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f04ac001850 con 0x7f04bc0b4500 2024-04-27T07:04:46.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 >> v1:172.21.15.12:6789/0 conn(0x7f04bc0b4500 legacy=0x7f04bc13f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 0x7f04a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a8004810 0x7f04a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc13fef0 0x7f04a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0b7ae0 con 0x7f04a80039f0 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04a800bab0 con 0x7f04a8004810 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f04a800bcb0 con 0x7f04bc13fef0 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04baffd640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 0x7f04a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04ba7fc640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a8004810 0x7f04a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04baffd640 1 -- 172.21.15.12:0/2728704278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc13fef0 msgr2=0x7f04a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04baffd640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc13fef0 0x7f04a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04baffd640 1 -- 172.21.15.12:0/2728704278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a8004810 msgr2=0x7f04a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04baffd640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04a8004810 0x7f04a8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.096+0000 7f04baffd640 1 -- 172.21.15.12:0/2728704278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a8003530 con 0x7f04a80039f0 2024-04-27T07:04:46.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04baffd640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 0x7f04a8003e10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c2f0 tx=0x7f04ac01cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38651e2f8badc560 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:46.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac007e10 con 0x7f04a80039f0 2024-04-27T07:04:46.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ac0a9070 con 0x7f04a80039f0 2024-04-27T07:04:46.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04b9ffb640 1 -- 172.21.15.12:0/2728704278 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac005d40 con 0x7f04a80039f0 2024-04-27T07:04:46.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- 172.21.15.12:0/2728704278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 msgr2=0x7f04a8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 --2- 172.21.15.12:0/2728704278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 0x7f04a8003e10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c2f0 tx=0x7f04ac01cb60 comp rx=0 tx=0).stop 2024-04-27T07:04:46.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04bb7fe640 1 -- 172.21.15.12:0/2728704278 reap_dead start 2024-04-27T07:04:46.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- 172.21.15.12:0/2728704278 shutdown_connections 2024-04-27T07:04:46.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- 172.21.15.12:0/2728704278 >> 172.21.15.12:0/2728704278 conn(0x7f04bc0bb4f0 msgr2=0x7f04bc0bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:46.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- 172.21.15.12:0/2728704278 shutdown_connections 2024-04-27T07:04:46.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- 172.21.15.12:0/2728704278 wait complete. 2024-04-27T07:04:46.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 Processor -- start 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- start start 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 0x7f04bc142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc1e18e0 0x7f04bc147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc1e1d40 0x7f04bc1481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc151db0 con 0x7f04a80039f0 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0b2c00 con 0x7f04bc1e18e0 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc151f30 con 0x7f04bc1e1d40 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04baffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 0x7f04bc142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04baffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 0x7f04bc142790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55094/0 (socket says 172.21.15.12:55094) 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04baffd640 1 -- 172.21.15.12:0/1903399694 learned_addr learned my addr 172.21.15.12:0/1903399694 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04ba7fc640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc1e18e0 0x7f04bc147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04bb7fe640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc1e1d40 0x7f04bc1481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04baffd640 1 -- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc1e1d40 msgr2=0x7f04bc1481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04baffd640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc1e1d40 0x7f04bc1481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04baffd640 1 -- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc1e18e0 msgr2=0x7f04bc147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04baffd640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc1e18e0 0x7f04bc147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04baffd640 1 -- 172.21.15.12:0/1903399694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc14a4d0 con 0x7f04a80039f0 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04ba7fc640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc1e18e0 0x7f04bc147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04bb7fe640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc1e1d40 0x7f04bc1481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04baffd640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 0x7f04bc142790 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f04ac01c5d0 tx=0x7f04a8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04b37fe640 1 -- 172.21.15.12:0/1903399694 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0047f0 con 0x7f04a80039f0 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04b37fe640 1 -- 172.21.15.12:0/1903399694 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ac004990 con 0x7f04a80039f0 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- 172.21.15.12:0/1903399694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc1e22e0 con 0x7f04a80039f0 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04b37fe640 1 -- 172.21.15.12:0/1903399694 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac004d40 con 0x7f04a80039f0 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.100+0000 7f04c13f8640 1 -- 172.21.15.12:0/1903399694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc14a150 con 0x7f04a80039f0 2024-04-27T07:04:46.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.104+0000 7f04c13f8640 1 -- 172.21.15.12:0/1903399694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc069330 con 0x7f04a80039f0 2024-04-27T07:04:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.104+0000 7f04b37fe640 1 -- 172.21.15.12:0/1903399694 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f04ac0b7030 con 0x7f04a80039f0 2024-04-27T07:04:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.104+0000 7f04b37fe640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f049c0683b0 0x7f049c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.104+0000 7f04b37fe640 1 -- 172.21.15.12:0/1903399694 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f04ac11ffa0 con 0x7f04a80039f0 2024-04-27T07:04:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.104+0000 7f04ba7fc640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f049c0683b0 0x7f049c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.104+0000 7f04ba7fc640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f049c0683b0 0x7f049c06a870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f04bc13f9c0 tx=0x7f04a400d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:46.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.108+0000 7f04b37fe640 1 -- 172.21.15.12:0/1903399694 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f04bc145a00 con 0x7f04a80039f0 2024-04-27T07:04:46.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.272+0000 7f04c13f8640 1 -- 172.21.15.12:0/1903399694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04bc1478b0 con 0x7f04a80039f0 2024-04-27T07:04:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04b37fe640 1 -- 172.21.15.12:0/1903399694 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f04ac117e20 con 0x7f04a80039f0 2024-04-27T07:04:46.282 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:46.282 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 -- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f049c0683b0 msgr2=0x7f049c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f049c0683b0 0x7f049c06a870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f04bc13f9c0 tx=0x7f04a400d040 comp rx=0 tx=0).stop 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 -- 172.21.15.12:0/1903399694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 msgr2=0x7f04bc142790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 0x7f04bc142790 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f04ac01c5d0 tx=0x7f04a8005840 comp rx=0 tx=0).stop 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 -- 172.21.15.12:0/1903399694 shutdown_connections 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f049c0683b0 0x7f049c06a870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04bc1e1d40 0x7f04bc1481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04bc1e18e0 0x7f04bc147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 --2- 172.21.15.12:0/1903399694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04a80039f0 0x7f04bc142790 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 -- 172.21.15.12:0/1903399694 >> 172.21.15.12:0/1903399694 conn(0x7f04bc0bb4f0 msgr2=0x7f04bc144030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 -- 172.21.15.12:0/1903399694 shutdown_connections 2024-04-27T07:04:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.276+0000 7f04c13f8640 1 -- 172.21.15.12:0/1903399694 wait complete. 2024-04-27T07:04:46.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:12.742740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:13.912898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:46.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:46.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94b254c640 1 Processor -- start 2024-04-27T07:04:46.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94b254c640 1 -- start start 2024-04-27T07:04:46.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94b254c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f94ac0be060 0x7f94ac14b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94b254c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ac0b4fb0 0x7f94ac14b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94b254c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94ac0be440 con 0x7f94ac0b5520 2024-04-27T07:04:46.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94b254c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f94ac0b5ef0 con 0x7f94ac0be060 2024-04-27T07:04:46.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94b254c640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f94ac0b6140 con 0x7f94ac0b4fb0 2024-04-27T07:04:46.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94ab7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f94ac0b5520 0x7f94ac141b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43222/0 (socket says 172.21.15.12:43222) 2024-04-27T07:04:46.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94abfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f94ac0be060 0x7f94ac14b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94ab7fe640 1 -- 172.21.15.12:0/64685638 learned_addr learned my addr 172.21.15.12:0/64685638 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:46.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94b0ae9640 1 --2- 172.21.15.12:0/64685638 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ac0b4fb0 0x7f94ac14b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94aaffd640 1 -- 172.21.15.12:0/64685638 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1035524996 0 0) 0x7f94ac0be440 con 0x7f94ac0b5520 2024-04-27T07:04:46.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94aaffd640 1 -- 172.21.15.12:0/64685638 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9494003530 con 0x7f94ac0b5520 2024-04-27T07:04:46.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94abfff640 1 -- 172.21.15.12:0/64685638 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ac0b4fb0 msgr2=0x7f94ac14b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:46.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94abfff640 1 --2- 172.21.15.12:0/64685638 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ac0b4fb0 0x7f94ac14b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94abfff640 1 -- 172.21.15.12:0/64685638 >> v1:172.21.15.12:6789/0 conn(0x7f94ac0b5520 legacy=0x7f94ac141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.420+0000 7f94abfff640 1 -- 172.21.15.12:0/64685638 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac0b6420 con 0x7f94ac0be060 2024-04-27T07:04:46.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94abfff640 1 --2- 172.21.15.12:0/64685638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac0be060 0x7f94ac14b3f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94a00097c0 tx=0x7f94a001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a107b7477f89cb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:46.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94aaffd640 1 -- 172.21.15.12:0/64685638 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a00b0070 con 0x7f94ac0be060 2024-04-27T07:04:46.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94aaffd640 1 -- 172.21.15.12:0/64685638 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a00ab600 con 0x7f94ac0be060 2024-04-27T07:04:46.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94aaffd640 1 -- 172.21.15.12:0/64685638 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a00ab9d0 con 0x7f94ac0be060 2024-04-27T07:04:46.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 -- 172.21.15.12:0/64685638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac0be060 msgr2=0x7f94ac14b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 --2- 172.21.15.12:0/64685638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac0be060 0x7f94ac14b3f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f94a00097c0 tx=0x7f94a001c410 comp rx=0 tx=0).stop 2024-04-27T07:04:46.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 -- 172.21.15.12:0/64685638 shutdown_connections 2024-04-27T07:04:46.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 --2- 172.21.15.12:0/64685638 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ac0b4fb0 0x7f94ac14b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 --2- 172.21.15.12:0/64685638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac0be060 0x7f94ac14b3f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 -- 172.21.15.12:0/64685638 >> 172.21.15.12:0/64685638 conn(0x7f94ac067f40 msgr2=0x7f94ac0bd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:46.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 -- 172.21.15.12:0/64685638 shutdown_connections 2024-04-27T07:04:46.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 -- 172.21.15.12:0/64685638 wait complete. 2024-04-27T07:04:46.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 Processor -- start 2024-04-27T07:04:46.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 -- start start 2024-04-27T07:04:46.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac0b4fb0 0x7f94ac147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ac0be060 0x7f94ac1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac148830 0x7f94ac13f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1552d0 con 0x7f94ac148830 2024-04-27T07:04:46.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac155150 con 0x7f94ac0b4fb0 2024-04-27T07:04:46.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b254c640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f94ac155450 con 0x7f94ac0be060 2024-04-27T07:04:46.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b0ae9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac148830 0x7f94ac13f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b0ae9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac148830 0x7f94ac13f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55114/0 (socket says 172.21.15.12:55114) 2024-04-27T07:04:46.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b0ae9640 1 -- 172.21.15.12:0/3836110314 learned_addr learned my addr 172.21.15.12:0/3836110314 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:46.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94ab7fe640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ac0be060 0x7f94ac1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94abfff640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac0b4fb0 0x7f94ac147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b0ae9640 1 -- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ac0be060 msgr2=0x7f94ac1482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b0ae9640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ac0be060 0x7f94ac1482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b0ae9640 1 -- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac0b4fb0 msgr2=0x7f94ac147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b0ae9640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac0b4fb0 0x7f94ac147490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.424+0000 7f94b0ae9640 1 -- 172.21.15.12:0/3836110314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac0b4220 con 0x7f94ac148830 2024-04-27T07:04:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94b0ae9640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac148830 0x7f94ac13f360 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f94a4007e40 tx=0x7f94a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94a8ff9640 1 -- 172.21.15.12:0/3836110314 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4014620 con 0x7f94ac148830 2024-04-27T07:04:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94a8ff9640 1 -- 172.21.15.12:0/3836110314 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a4019070 con 0x7f94ac148830 2024-04-27T07:04:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94b254c640 1 -- 172.21.15.12:0/3836110314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac145cf0 con 0x7f94ac148830 2024-04-27T07:04:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94a8ff9640 1 -- 172.21.15.12:0/3836110314 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4014a20 con 0x7f94ac148830 2024-04-27T07:04:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94b254c640 1 -- 172.21.15.12:0/3836110314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac13f8a0 con 0x7f94ac148830 2024-04-27T07:04:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94b254c640 1 -- 172.21.15.12:0/3836110314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac069330 con 0x7f94ac148830 2024-04-27T07:04:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94a8ff9640 1 -- 172.21.15.12:0/3836110314 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f94a4014bc0 con 0x7f94ac148830 2024-04-27T07:04:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94a8ff9640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f94980683b0 0x7f949806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94a8ff9640 1 -- 172.21.15.12:0/3836110314 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f94a4087190 con 0x7f94ac148830 2024-04-27T07:04:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94abfff640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f94980683b0 0x7f949806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.428+0000 7f94abfff640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f94980683b0 0x7f949806a870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94a00ad000 tx=0x7f94a00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.432+0000 7f94a8ff9640 1 -- 172.21.15.12:0/3836110314 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f94a40577e0 con 0x7f94ac148830 2024-04-27T07:04:46.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.588+0000 7f94b254c640 1 -- 172.21.15.12:0/3836110314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f94ac140780 con 0x7f94ac148830 2024-04-27T07:04:46.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94a8ff9640 1 -- 172.21.15.12:0/3836110314 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f94a40570e0 con 0x7f94ac148830 2024-04-27T07:04:46.597 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:46.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 -- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f94980683b0 msgr2=0x7f949806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f94980683b0 0x7f949806a870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94a00ad000 tx=0x7f94a00a6000 comp rx=0 tx=0).stop 2024-04-27T07:04:46.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 -- 172.21.15.12:0/3836110314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac148830 msgr2=0x7f94ac13f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac148830 0x7f94ac13f360 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f94a4007e40 tx=0x7f94a400e4b0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 -- 172.21.15.12:0/3836110314 shutdown_connections 2024-04-27T07:04:46.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f94980683b0 0x7f949806a870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94ac148830 0x7f94ac13f360 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f94ac0be060 0x7f94ac1482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 --2- 172.21.15.12:0/3836110314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f94ac0b4fb0 0x7f94ac147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 -- 172.21.15.12:0/3836110314 >> 172.21.15.12:0/3836110314 conn(0x7f94ac067f40 msgr2=0x7f94ac0b87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 -- 172.21.15.12:0/3836110314 shutdown_connections 2024-04-27T07:04:46.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.592+0000 7f94b254c640 1 -- 172.21.15.12:0/3836110314 wait complete. 2024-04-27T07:04:46.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:04:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b3a27640 1 Processor -- start 2024-04-27T07:04:46.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b3a27640 1 -- start start 2024-04-27T07:04:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b3a27640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac0b4660 0x7f91ac151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b3a27640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91ac0b4af0 con 0x7f91ac0b4220 2024-04-27T07:04:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b3a27640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91ac0b5850 con 0x7f91ac0b5470 2024-04-27T07:04:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b3a27640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f91ac0b29f0 con 0x7f91ac0b4660 2024-04-27T07:04:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b0fc2640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f91ac0b4220 0x7f91ac14e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43236/0 (socket says 172.21.15.12:43236) 2024-04-27T07:04:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b0fc2640 1 -- 172.21.15.12:0/1017646132 learned_addr learned my addr 172.21.15.12:0/1017646132 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b1fc4640 1 --2- 172.21.15.12:0/1017646132 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac0b4660 0x7f91ac151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2071904165 0 0) 0x7f91ac0b4af0 con 0x7f91ac0b4220 2024-04-27T07:04:46.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9198003530 con 0x7f91ac0b4220 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2215390268 0 0) 0x7f9198003530 con 0x7f91ac0b4220 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac0b4660 msgr2=0x7f91ac151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 --2- 172.21.15.12:0/1017646132 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac0b4660 0x7f91ac151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 >> v1:172.21.15.53:6789/0 conn(0x7f91ac0b5470 legacy=0x7f91ac13fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac0b2c40 con 0x7f91ac0b4220 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f919c001680 con 0x7f91ac0b4220 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 >> v1:172.21.15.12:6789/0 conn(0x7f91ac0b4220 legacy=0x7f91ac14e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 --2- 172.21.15.12:0/1017646132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91980039f0 0x7f9198003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 --2- 172.21.15.12:0/1017646132 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac0b4660 0x7f9198004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 --2- 172.21.15.12:0/1017646132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9198004d50 0x7f91980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac0b29f0 con 0x7f91980039f0 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f919800bb00 con 0x7f9198004d50 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f919800bd00 con 0x7f91ac0b4660 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b17c3640 1 --2- 172.21.15.12:0/1017646132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91980039f0 0x7f9198003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b17c3640 1 -- 172.21.15.12:0/1017646132 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac0b4660 msgr2=0x7f9198004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b17c3640 1 --2- 172.21.15.12:0/1017646132 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac0b4660 0x7f9198004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b17c3640 1 -- 172.21.15.12:0/1017646132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9198004d50 msgr2=0x7f91980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b17c3640 1 --2- 172.21.15.12:0/1017646132 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9198004d50 0x7f91980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.736+0000 7f91b17c3640 1 -- 172.21.15.12:0/1017646132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9198003530 con 0x7f91980039f0 2024-04-27T07:04:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b17c3640 1 --2- 172.21.15.12:0/1017646132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91980039f0 0x7f9198003e10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f91a8001780 tx=0x7f91a801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e57f23986613af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a80013a0 con 0x7f91980039f0 2024-04-27T07:04:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a80052f0 con 0x7f91980039f0 2024-04-27T07:04:46.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91a3fff640 1 -- 172.21.15.12:0/1017646132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8003840 con 0x7f91980039f0 2024-04-27T07:04:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 -- 172.21.15.12:0/1017646132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91980039f0 msgr2=0x7f9198003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 --2- 172.21.15.12:0/1017646132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91980039f0 0x7f9198003e10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f91a8001780 tx=0x7f91a801c360 comp rx=0 tx=0).stop 2024-04-27T07:04:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b1fc4640 1 -- 172.21.15.12:0/1017646132 reap_dead start 2024-04-27T07:04:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 -- 172.21.15.12:0/1017646132 shutdown_connections 2024-04-27T07:04:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 -- 172.21.15.12:0/1017646132 >> 172.21.15.12:0/1017646132 conn(0x7f91ac068210 msgr2=0x7f91ac0bb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 -- 172.21.15.12:0/1017646132 shutdown_connections 2024-04-27T07:04:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 -- 172.21.15.12:0/1017646132 wait complete. 2024-04-27T07:04:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 Processor -- start 2024-04-27T07:04:46.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 -- start start 2024-04-27T07:04:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac142a80 0x7f91ac1de6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91ac14a5a0 0x7f91ac1e1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91ac1debf0 0x7f91ac1e2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac140450 con 0x7f91ac14a5a0 2024-04-27T07:04:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1402d0 con 0x7f91ac1debf0 2024-04-27T07:04:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b3a27640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f91ac0b7330 con 0x7f91ac142a80 2024-04-27T07:04:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b0fc2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91ac14a5a0 0x7f91ac1e1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b0fc2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91ac14a5a0 0x7f91ac1e1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55132/0 (socket says 172.21.15.12:55132) 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b0fc2640 1 -- 172.21.15.12:0/394220763 learned_addr learned my addr 172.21.15.12:0/394220763 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b17c3640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac142a80 0x7f91ac1de6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b1fc4640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91ac1debf0 0x7f91ac1e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b0fc2640 1 -- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac142a80 msgr2=0x7f91ac1de6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b0fc2640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac142a80 0x7f91ac1de6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b0fc2640 1 -- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91ac1debf0 msgr2=0x7f91ac1e2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b0fc2640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91ac1debf0 0x7f91ac1e2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b0fc2640 1 -- 172.21.15.12:0/394220763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac142e60 con 0x7f91ac14a5a0 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b1fc4640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91ac1debf0 0x7f91ac1e2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.740+0000 7f91b0fc2640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91ac14a5a0 0x7f91ac1e1c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f919c00c760 tx=0x7f919c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91a1ffb640 1 -- 172.21.15.12:0/394220763 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c018070 con 0x7f91ac14a5a0 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91a1ffb640 1 -- 172.21.15.12:0/394220763 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919c007520 con 0x7f91ac14a5a0 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91a1ffb640 1 -- 172.21.15.12:0/394220763 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c004a30 con 0x7f91ac14a5a0 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91b3a27640 1 -- 172.21.15.12:0/394220763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ac14b780 con 0x7f91ac14a5a0 2024-04-27T07:04:46.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91b3a27640 1 -- 172.21.15.12:0/394220763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ac149da0 con 0x7f91ac14a5a0 2024-04-27T07:04:46.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91b3a27640 1 -- 172.21.15.12:0/394220763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ac069330 con 0x7f91ac14a5a0 2024-04-27T07:04:46.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91a1ffb640 1 -- 172.21.15.12:0/394220763 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f919c004bd0 con 0x7f91ac14a5a0 2024-04-27T07:04:46.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91a1ffb640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9190068380 0x7f919006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:46.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91b17c3640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9190068380 0x7f919006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:46.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91a1ffb640 1 -- 172.21.15.12:0/394220763 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f919c053e30 con 0x7f91ac14a5a0 2024-04-27T07:04:46.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.744+0000 7f91b17c3640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9190068380 0x7f919006a840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91a8001750 tx=0x7f91a80011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:46.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.752+0000 7f91a1ffb640 1 -- 172.21.15.12:0/394220763 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f919c055d90 con 0x7f91ac14a5a0 2024-04-27T07:04:46.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.904+0000 7f91b3a27640 1 -- 172.21.15.12:0/394220763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f91ac0b37d0 con 0x7f91ac14a5a0 2024-04-27T07:04:46.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.904+0000 7f91a1ffb640 1 -- 172.21.15.12:0/394220763 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f919c05a090 con 0x7f91ac14a5a0 2024-04-27T07:04:46.911 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 -- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9190068380 msgr2=0x7f919006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9190068380 0x7f919006a840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91a8001750 tx=0x7f91a80011a0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 -- 172.21.15.12:0/394220763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91ac14a5a0 msgr2=0x7f91ac1e1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91ac14a5a0 0x7f91ac1e1c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f919c00c760 tx=0x7f919c00cc20 comp rx=0 tx=0).stop 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 -- 172.21.15.12:0/394220763 shutdown_connections 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9190068380 0x7f919006a840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91ac1debf0 0x7f91ac1e2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91ac14a5a0 0x7f91ac1e1c20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 --2- 172.21.15.12:0/394220763 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f91ac142a80 0x7f91ac1de6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 -- 172.21.15.12:0/394220763 >> 172.21.15.12:0/394220763 conn(0x7f91ac068210 msgr2=0x7f91ac13de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 -- 172.21.15.12:0/394220763 shutdown_connections 2024-04-27T07:04:46.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:46.908+0000 7f91b3a27640 1 -- 172.21.15.12:0/394220763 wait complete. 2024-04-27T07:04:46.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:04:47.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88e1dad640 1 Processor -- start 2024-04-27T07:04:47.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88e1dad640 1 -- start start 2024-04-27T07:04:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88e1dad640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f88dc06e370 0x7f88dc13faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88e1dad640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88dc0b4d70 con 0x7f88dc0b3650 2024-04-27T07:04:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88e1dad640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88dc073e60 con 0x7f88dc06d300 2024-04-27T07:04:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88e1dad640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f88dc0740d0 con 0x7f88dc06e370 2024-04-27T07:04:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88daffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f88dc0b3650 0x7f88dc0b5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43244/0 (socket says 172.21.15.12:43244) 2024-04-27T07:04:47.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88db7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f88dc06e370 0x7f88dc13faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88daffd640 1 -- 172.21.15.12:0/2823378129 learned_addr learned my addr 172.21.15.12:0/2823378129 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:47.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88da7fc640 1 -- 172.21.15.12:0/2823378129 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2134287498 0 0) 0x7f88dc0b4d70 con 0x7f88dc0b3650 2024-04-27T07:04:47.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88da7fc640 1 -- 172.21.15.12:0/2823378129 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88c4003530 con 0x7f88dc0b3650 2024-04-27T07:04:47.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88db7fe640 1 -- 172.21.15.12:0/2823378129 >> v1:172.21.15.53:6789/0 conn(0x7f88dc06d300 legacy=0x7f88dc150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:47.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88db7fe640 1 -- 172.21.15.12:0/2823378129 >> v1:172.21.15.12:6789/0 conn(0x7f88dc0b3650 legacy=0x7f88dc0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.040+0000 7f88db7fe640 1 -- 172.21.15.12:0/2823378129 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc0b2bf0 con 0x7f88dc06e370 2024-04-27T07:04:47.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88db7fe640 1 --2- 172.21.15.12:0/2823378129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06e370 0x7f88dc13faa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0097c0 tx=0x7f88cc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8a0a99b756af27 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:47.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88da7fc640 1 -- 172.21.15.12:0/2823378129 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0b0070 con 0x7f88dc06e370 2024-04-27T07:04:47.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88da7fc640 1 -- 172.21.15.12:0/2823378129 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88cc0ab600 con 0x7f88dc06e370 2024-04-27T07:04:47.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88da7fc640 1 -- 172.21.15.12:0/2823378129 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0ab9d0 con 0x7f88dc06e370 2024-04-27T07:04:47.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 -- 172.21.15.12:0/2823378129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06e370 msgr2=0x7f88dc13faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 --2- 172.21.15.12:0/2823378129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06e370 0x7f88dc13faa0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0097c0 tx=0x7f88cc01c3d0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 -- 172.21.15.12:0/2823378129 shutdown_connections 2024-04-27T07:04:47.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 --2- 172.21.15.12:0/2823378129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06e370 0x7f88dc13faa0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 -- 172.21.15.12:0/2823378129 >> 172.21.15.12:0/2823378129 conn(0x7f88dc0b93b0 msgr2=0x7f88dc0b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:47.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 -- 172.21.15.12:0/2823378129 shutdown_connections 2024-04-27T07:04:47.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 -- 172.21.15.12:0/2823378129 wait complete. 2024-04-27T07:04:47.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 Processor -- start 2024-04-27T07:04:47.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 -- start start 2024-04-27T07:04:47.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06d300 0x7f88dc071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88dc06e370 0x7f88dc1df4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88dc0b3650 0x7f88dc145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1546f0 con 0x7f88dc06d300 2024-04-27T07:04:47.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc154570 con 0x7f88dc0b3650 2024-04-27T07:04:47.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88e1dad640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f88dc154870 con 0x7f88dc06e370 2024-04-27T07:04:47.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88db7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06d300 0x7f88dc071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88db7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06d300 0x7f88dc071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55164/0 (socket says 172.21.15.12:55164) 2024-04-27T07:04:47.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.044+0000 7f88db7fe640 1 -- 172.21.15.12:0/2808528845 learned_addr learned my addr 172.21.15.12:0/2808528845 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:47.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88daffd640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88dc06e370 0x7f88dc1df4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88dbfff640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88dc0b3650 0x7f88dc145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88db7fe640 1 -- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88dc06e370 msgr2=0x7f88dc1df4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88db7fe640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88dc06e370 0x7f88dc1df4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88db7fe640 1 -- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88dc0b3650 msgr2=0x7f88dc145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88db7fe640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88dc0b3650 0x7f88dc145680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88db7fe640 1 -- 172.21.15.12:0/2808528845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc1de9e0 con 0x7f88dc06d300 2024-04-27T07:04:47.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88daffd640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88dc06e370 0x7f88dc1df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:47.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88db7fe640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06d300 0x7f88dc071590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0097c0 tx=0x7f88cc01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:47.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88c3fff640 1 -- 172.21.15.12:0/2808528845 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0b0050 con 0x7f88dc06d300 2024-04-27T07:04:47.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88c3fff640 1 -- 172.21.15.12:0/2808528845 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88cc0bc070 con 0x7f88dc06d300 2024-04-27T07:04:47.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88e1dad640 1 -- 172.21.15.12:0/2808528845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc0730e0 con 0x7f88dc06d300 2024-04-27T07:04:47.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88c3fff640 1 -- 172.21.15.12:0/2808528845 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0b5830 con 0x7f88dc06d300 2024-04-27T07:04:47.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88e1dad640 1 -- 172.21.15.12:0/2808528845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc1def80 con 0x7f88dc06d300 2024-04-27T07:04:47.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88e1dad640 1 -- 172.21.15.12:0/2808528845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88dc140720 con 0x7f88dc06d300 2024-04-27T07:04:47.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88c3fff640 1 -- 172.21.15.12:0/2808528845 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f88cc0b59d0 con 0x7f88dc06d300 2024-04-27T07:04:47.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88c3fff640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f88c8068380 0x7f88c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88c3fff640 1 -- 172.21.15.12:0/2808528845 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f88cc121ac0 con 0x7f88dc06d300 2024-04-27T07:04:47.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88daffd640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f88c8068380 0x7f88c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.048+0000 7f88daffd640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f88c8068380 0x7f88c806a840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f88dc09e480 tx=0x7f88d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:47.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.052+0000 7f88c3fff640 1 -- 172.21.15.12:0/2808528845 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f88dc140720 con 0x7f88dc06d300 2024-04-27T07:04:47.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 -- 172.21.15.12:0/2808528845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f88dc143410 con 0x7f88dc06d300 2024-04-27T07:04:47.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88c3fff640 1 -- 172.21.15.12:0/2808528845 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f88cc0f2e20 con 0x7f88dc06d300 2024-04-27T07:04:47.217 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 -- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f88c8068380 msgr2=0x7f88c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f88c8068380 0x7f88c806a840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f88dc09e480 tx=0x7f88d0009040 comp rx=0 tx=0).stop 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 -- 172.21.15.12:0/2808528845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06d300 msgr2=0x7f88dc071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06d300 0x7f88dc071590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0097c0 tx=0x7f88cc01ac10 comp rx=0 tx=0).stop 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 -- 172.21.15.12:0/2808528845 shutdown_connections 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f88c8068380 0x7f88c806a840 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f88dc0b3650 0x7f88dc145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f88dc06e370 0x7f88dc1df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 --2- 172.21.15.12:0/2808528845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88dc06d300 0x7f88dc071590 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 -- 172.21.15.12:0/2808528845 >> 172.21.15.12:0/2808528845 conn(0x7f88dc0b93b0 msgr2=0x7f88dc13d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 -- 172.21.15.12:0/2808528845 shutdown_connections 2024-04-27T07:04:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.212+0000 7f88e1dad640 1 -- 172.21.15.12:0/2808528845 wait complete. 2024-04-27T07:04:47.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:04:47.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe29e381640 1 Processor -- start 2024-04-27T07:04:47.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe29e381640 1 -- start start 2024-04-27T07:04:47.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe29e381640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe29806e370 0x7fe29813faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe29e381640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 0x7fe298150420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe29e381640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2980b4d70 con 0x7fe2980b3650 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe29e381640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fe2980740d0 con 0x7fe29806e370 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe29e381640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe2980b2bf0 con 0x7fe29806d300 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe297fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe29806e370 0x7fe29813faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe297fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe29806e370 0x7fe29813faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55168/0 (socket says 172.21.15.12:55168) 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe297fff640 1 -- 172.21.15.12:0/68301082 learned_addr learned my addr 172.21.15.12:0/68301082 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe29c91e640 1 --2- 172.21.15.12:0/68301082 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 0x7fe298150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe297fff640 1 -- 172.21.15.12:0/68301082 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 msgr2=0x7fe298150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe297fff640 1 --2- 172.21.15.12:0/68301082 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 0x7fe298150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe297fff640 1 -- 172.21.15.12:0/68301082 >> v1:172.21.15.53:6789/0 conn(0x7fe2980b3650 legacy=0x7fe2980b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:47.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.340+0000 7fe297fff640 1 -- 172.21.15.12:0/68301082 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe298073e60 con 0x7fe29806e370 2024-04-27T07:04:47.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe297fff640 1 --2- 172.21.15.12:0/68301082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe29806e370 0x7fe29813faa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe290009870 tx=0x7fe29001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf3486d6866bb4a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:47.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe296ffd640 1 -- 172.21.15.12:0/68301082 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2900b0070 con 0x7fe29806e370 2024-04-27T07:04:47.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe296ffd640 1 -- 172.21.15.12:0/68301082 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2900ab600 con 0x7fe29806e370 2024-04-27T07:04:47.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe296ffd640 1 -- 172.21.15.12:0/68301082 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2900ab9d0 con 0x7fe29806e370 2024-04-27T07:04:47.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 -- 172.21.15.12:0/68301082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe29806e370 msgr2=0x7fe29813faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 --2- 172.21.15.12:0/68301082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe29806e370 0x7fe29813faa0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe290009870 tx=0x7fe29001c490 comp rx=0 tx=0).stop 2024-04-27T07:04:47.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 -- 172.21.15.12:0/68301082 shutdown_connections 2024-04-27T07:04:47.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 --2- 172.21.15.12:0/68301082 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 0x7fe298150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 --2- 172.21.15.12:0/68301082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe29806e370 0x7fe29813faa0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 -- 172.21.15.12:0/68301082 >> 172.21.15.12:0/68301082 conn(0x7fe2980b93b0 msgr2=0x7fe2980b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:47.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 -- 172.21.15.12:0/68301082 shutdown_connections 2024-04-27T07:04:47.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 -- 172.21.15.12:0/68301082 wait complete. 2024-04-27T07:04:47.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 Processor -- start 2024-04-27T07:04:47.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 -- start start 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 0x7fe298143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe29806e370 0x7fe298145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2980b3650 0x7fe2981461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe298153510 con 0x7fe2980b3650 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe298153390 con 0x7fe29806e370 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.344+0000 7fe29e381640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe298153690 con 0x7fe29806d300 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29c91e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2980b3650 0x7fe2981461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29c91e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2980b3650 0x7fe2981461f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55182/0 (socket says 172.21.15.12:55182) 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29c91e640 1 -- 172.21.15.12:0/628896555 learned_addr learned my addr 172.21.15.12:0/628896555 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe297fff640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 0x7fe298143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe2977fe640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe29806e370 0x7fe298145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29c91e640 1 -- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 msgr2=0x7fe298143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29c91e640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 0x7fe298143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29c91e640 1 -- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe29806e370 msgr2=0x7fe298145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29c91e640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe29806e370 0x7fe298145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29c91e640 1 -- 172.21.15.12:0/628896555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29809ddc0 con 0x7fe2980b3650 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe297fff640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 0x7fe298143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe2977fe640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe29806e370 0x7fe298145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29c91e640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2980b3650 0x7fe2981461f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe288007de0 tx=0x7fe28800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe294ff9640 1 -- 172.21.15.12:0/628896555 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288014620 con 0x7fe2980b3650 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe294ff9640 1 -- 172.21.15.12:0/628896555 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe288019070 con 0x7fe2980b3650 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29e381640 1 -- 172.21.15.12:0/628896555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe298145140 con 0x7fe2980b3650 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe294ff9640 1 -- 172.21.15.12:0/628896555 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288014a20 con 0x7fe2980b3650 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29e381640 1 -- 172.21.15.12:0/628896555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe298144e60 con 0x7fe2980b3650 2024-04-27T07:04:47.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe29e381640 1 -- 172.21.15.12:0/628896555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe298068c20 con 0x7fe2980b3650 2024-04-27T07:04:47.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe294ff9640 1 -- 172.21.15.12:0/628896555 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe288014bc0 con 0x7fe2980b3650 2024-04-27T07:04:47.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe294ff9640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe278068380 0x7fe27806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe294ff9640 1 -- 172.21.15.12:0/628896555 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe2880872a0 con 0x7fe2980b3650 2024-04-27T07:04:47.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe297fff640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe278068380 0x7fe27806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.348+0000 7fe297fff640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe278068380 0x7fe27806a840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe2900ad000 tx=0x7fe290008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:47.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.352+0000 7fe294ff9640 1 -- 172.21.15.12:0/628896555 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe2880578f0 con 0x7fe2980b3650 2024-04-27T07:04:47.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 -- 172.21.15.12:0/628896555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe2980afa70 con 0x7fe2980b3650 2024-04-27T07:04:47.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe294ff9640 1 -- 172.21.15.12:0/628896555 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fe2880571f0 con 0x7fe2980b3650 2024-04-27T07:04:47.533 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2024-04-27T07:04:47.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 -- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe278068380 msgr2=0x7fe27806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe278068380 0x7fe27806a840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe2900ad000 tx=0x7fe290008040 comp rx=0 tx=0).stop 2024-04-27T07:04:47.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 -- 172.21.15.12:0/628896555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2980b3650 msgr2=0x7fe2981461f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2980b3650 0x7fe2981461f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe288007de0 tx=0x7fe28800e4b0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 -- 172.21.15.12:0/628896555 shutdown_connections 2024-04-27T07:04:47.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe278068380 0x7fe27806a840 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2980b3650 0x7fe2981461f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe29806e370 0x7fe298145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 --2- 172.21.15.12:0/628896555 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe29806d300 0x7fe298143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 -- 172.21.15.12:0/628896555 >> 172.21.15.12:0/628896555 conn(0x7fe2980b93b0 msgr2=0x7fe298074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:47.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 -- 172.21.15.12:0/628896555 shutdown_connections 2024-04-27T07:04:47.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.528+0000 7fe29e381640 1 -- 172.21.15.12:0/628896555 wait complete. 2024-04-27T07:04:47.551 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi012.front.sepia.ceph.com... 2024-04-27T07:04:47.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:47.552 DEBUG:teuthology.orchestra.run.smithi012:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-27T07:04:47.577 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:04:47.578 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:04:47.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:04:47.578 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-27T07:04:47.633 INFO:tasks.cephfs.cephfs_test_case:[] 2024-04-27T07:04:47.633 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:04:47.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b92cb640 1 Processor -- start 2024-04-27T07:04:47.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b92cb640 1 -- start start 2024-04-27T07:04:47.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b92cb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe0b40731d0 0x7fe0b413f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b92cb640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0b40b3630 0x7fe0b413fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b92cb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0b40735b0 con 0x7fe0b40b11b0 2024-04-27T07:04:47.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b92cb640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fe0b40b3f00 con 0x7fe0b40731d0 2024-04-27T07:04:47.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b92cb640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b40b3c20 con 0x7fe0b40b3630 2024-04-27T07:04:47.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b2ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe0b40731d0 0x7fe0b413f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b2ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe0b40731d0 0x7fe0b413f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46706/0 (socket says 172.21.15.12:46706) 2024-04-27T07:04:47.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b2ffd640 1 -- 172.21.15.12:0/2266285741 learned_addr learned my addr 172.21.15.12:0/2266285741 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:47.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b37fe640 1 --2- 172.21.15.12:0/2266285741 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0b40b3630 0x7fe0b413fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b2ffd640 1 -- 172.21.15.12:0/2266285741 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0b40b3630 msgr2=0x7fe0b413fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:47.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b2ffd640 1 --2- 172.21.15.12:0/2266285741 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0b40b3630 0x7fe0b413fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b2ffd640 1 -- 172.21.15.12:0/2266285741 >> v1:172.21.15.53:6789/0 conn(0x7fe0b40b11b0 legacy=0x7fe0b40b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:47.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.796+0000 7fe0b2ffd640 1 -- 172.21.15.12:0/2266285741 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b40b40d0 con 0x7fe0b40731d0 2024-04-27T07:04:47.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b2ffd640 1 --2- 172.21.15.12:0/2266285741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40731d0 0x7fe0b413f7c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009870 tx=0x7fe0a401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e4d62f7bc08b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:47.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b1ffb640 1 -- 172.21.15.12:0/2266285741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40b0070 con 0x7fe0b40731d0 2024-04-27T07:04:47.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b1ffb640 1 -- 172.21.15.12:0/2266285741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a40ab600 con 0x7fe0b40731d0 2024-04-27T07:04:47.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b1ffb640 1 -- 172.21.15.12:0/2266285741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40ab9d0 con 0x7fe0b40731d0 2024-04-27T07:04:47.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 -- 172.21.15.12:0/2266285741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40731d0 msgr2=0x7fe0b413f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 --2- 172.21.15.12:0/2266285741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40731d0 0x7fe0b413f7c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009870 tx=0x7fe0a401c400 comp rx=0 tx=0).stop 2024-04-27T07:04:47.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 -- 172.21.15.12:0/2266285741 shutdown_connections 2024-04-27T07:04:47.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 --2- 172.21.15.12:0/2266285741 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0b40b3630 0x7fe0b413fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 --2- 172.21.15.12:0/2266285741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40731d0 0x7fe0b413f7c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 -- 172.21.15.12:0/2266285741 >> 172.21.15.12:0/2266285741 conn(0x7fe0b40b9e80 msgr2=0x7fe0b40ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:47.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 -- 172.21.15.12:0/2266285741 shutdown_connections 2024-04-27T07:04:47.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 -- 172.21.15.12:0/2266285741 wait complete. 2024-04-27T07:04:47.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 Processor -- start 2024-04-27T07:04:47.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 -- start start 2024-04-27T07:04:47.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0b40731d0 0x7fe0b41438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40b11b0 0x7fe0b40b9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0b40b3630 0x7fe0b41e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b4152fa0 con 0x7fe0b40b11b0 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b4152e20 con 0x7fe0b40b3630 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b92cb640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b4153120 con 0x7fe0b40731d0 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40b11b0 0x7fe0b40b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40b11b0 0x7fe0b40b9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46712/0 (socket says 172.21.15.12:46712) 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b27fc640 1 -- 172.21.15.12:0/55735783 learned_addr learned my addr 172.21.15.12:0/55735783 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b37fe640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0b40b3630 0x7fe0b41e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b2ffd640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0b40731d0 0x7fe0b41438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b27fc640 1 -- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0b40731d0 msgr2=0x7fe0b41438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b27fc640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0b40731d0 0x7fe0b41438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b27fc640 1 -- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0b40b3630 msgr2=0x7fe0b41e3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b27fc640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0b40b3630 0x7fe0b41e3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b27fc640 1 -- 172.21.15.12:0/55735783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b4146470 con 0x7fe0b40b11b0 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.800+0000 7fe0b27fc640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40b11b0 0x7fe0b40b9300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800c410 tx=0x7fe0a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe09b7fe640 1 -- 172.21.15.12:0/55735783 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8011790 con 0x7fe0b40b11b0 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe09b7fe640 1 -- 172.21.15.12:0/55735783 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a8017070 con 0x7fe0b40b11b0 2024-04-27T07:04:47.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe0b92cb640 1 -- 172.21.15.12:0/55735783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b4144780 con 0x7fe0b40b11b0 2024-04-27T07:04:47.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe09b7fe640 1 -- 172.21.15.12:0/55735783 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8011b70 con 0x7fe0b40b11b0 2024-04-27T07:04:47.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe0b92cb640 1 -- 172.21.15.12:0/55735783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b4071e00 con 0x7fe0b40b11b0 2024-04-27T07:04:47.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe0b92cb640 1 -- 172.21.15.12:0/55735783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b4069330 con 0x7fe0b40b11b0 2024-04-27T07:04:47.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe09b7fe640 1 -- 172.21.15.12:0/55735783 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe0a8011d10 con 0x7fe0b40b11b0 2024-04-27T07:04:47.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe09b7fe640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe0a0068380 0x7fe0a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:47.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe09b7fe640 1 -- 172.21.15.12:0/55735783 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe0a80865a0 con 0x7fe0b40b11b0 2024-04-27T07:04:47.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe0b2ffd640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe0a0068380 0x7fe0a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:47.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.804+0000 7fe0b2ffd640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe0a0068380 0x7fe0a006a840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40ad000 tx=0x7fe0a40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:47.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:47.808+0000 7fe09b7fe640 1 -- 172.21.15.12:0/55735783 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe0a8021020 con 0x7fe0b40b11b0 2024-04-27T07:04:48.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.004+0000 7fe0b92cb640 1 -- 172.21.15.12:0/55735783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0b4146a10 con 0x7fe0b40b11b0 2024-04-27T07:04:48.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe09b7fe640 1 -- 172.21.15.12:0/55735783 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3527 (secure 0 0 0) 0x7fe0a8056bf0 con 0x7fe0b40b11b0 2024-04-27T07:04:48.012 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:48.012 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1908224930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1908224930},{"type":"v1","addr":"172.21.15.53:6835","nonce":1908224930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:04:14.961460+0000","modified":"2024-04-27T07:04:17.982838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4242":{"gid":4242,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4173401303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4173401303},{"type":"v1","addr":"172.21.15.12:6833","nonce":4173401303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4242,"qdb_cluster":[4242]},"id":1}]} 2024-04-27T07:04:48.013 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-04-27T07:04:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 -- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe0a0068380 msgr2=0x7fe0a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe0a0068380 0x7fe0a006a840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40ad000 tx=0x7fe0a40a6000 comp rx=0 tx=0).stop 2024-04-27T07:04:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 -- 172.21.15.12:0/55735783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40b11b0 msgr2=0x7fe0b40b9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40b11b0 0x7fe0b40b9300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800c410 tx=0x7fe0a800c8d0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 -- 172.21.15.12:0/55735783 shutdown_connections 2024-04-27T07:04:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe0a0068380 0x7fe0a006a840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0b40b3630 0x7fe0b41e3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0b40b11b0 0x7fe0b40b9300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 --2- 172.21.15.12:0/55735783 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe0b40731d0 0x7fe0b41438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 -- 172.21.15.12:0/55735783 >> 172.21.15.12:0/55735783 conn(0x7fe0b40b9e80 msgr2=0x7fe0b4138f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 -- 172.21.15.12:0/55735783 shutdown_connections 2024-04-27T07:04:48.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.008+0000 7fe0b92cb640 1 -- 172.21.15.12:0/55735783 wait complete. 2024-04-27T07:04:48.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef7d3c2640 1 Processor -- start 2024-04-27T07:04:48.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef7d3c2640 1 -- start start 2024-04-27T07:04:48.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef7d3c2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fef7806e500 0x7fef780738a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef7d3c2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 0x7fef78147db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef7d3c2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef7806e8e0 0x7fef781482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef7d3c2640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fef780b24a0 con 0x7fef7806e500 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef7d3c2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fef780b2130 con 0x7fef7806e8e0 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef7d3c2640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fef780b7b40 con 0x7fef7806cc90 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef76ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fef7806e500 0x7fef780738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef76ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fef7806e500 0x7fef780738a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46724/0 (socket says 172.21.15.12:46724) 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef76ffd640 1 -- 172.21.15.12:0/271096089 learned_addr learned my addr 172.21.15.12:0/271096089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef767fc640 1 --2- 172.21.15.12:0/271096089 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 0x7fef78147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef777fe640 1 --2- 172.21.15.12:0/271096089 >> v2:172.21.15.53:3300/0 conn(0x7fef7806e8e0 0x7fef781482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef76ffd640 1 -- 172.21.15.12:0/271096089 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 msgr2=0x7fef78147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef76ffd640 1 --2- 172.21.15.12:0/271096089 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 0x7fef78147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef76ffd640 1 -- 172.21.15.12:0/271096089 >> v2:172.21.15.53:3300/0 conn(0x7fef7806e8e0 msgr2=0x7fef781482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef76ffd640 1 --2- 172.21.15.12:0/271096089 >> v2:172.21.15.53:3300/0 conn(0x7fef7806e8e0 0x7fef781482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef76ffd640 1 -- 172.21.15.12:0/271096089 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef780b5840 con 0x7fef7806e500 2024-04-27T07:04:48.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.148+0000 7fef777fe640 1 --2- 172.21.15.12:0/271096089 >> v2:172.21.15.53:3300/0 conn(0x7fef7806e8e0 0x7fef781482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:48.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef76ffd640 1 --2- 172.21.15.12:0/271096089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 0x7fef780738a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fef78148dc0 tx=0x7fef6001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404683d229d7d93b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:48.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef75ffb640 1 -- 172.21.15.12:0/271096089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef600b2070 con 0x7fef7806e500 2024-04-27T07:04:48.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef75ffb640 1 -- 172.21.15.12:0/271096089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef600a9070 con 0x7fef7806e500 2024-04-27T07:04:48.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef75ffb640 1 -- 172.21.15.12:0/271096089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef600ad880 con 0x7fef7806e500 2024-04-27T07:04:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 -- 172.21.15.12:0/271096089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 msgr2=0x7fef780738a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 --2- 172.21.15.12:0/271096089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 0x7fef780738a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fef78148dc0 tx=0x7fef6001c490 comp rx=0 tx=0).stop 2024-04-27T07:04:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 -- 172.21.15.12:0/271096089 shutdown_connections 2024-04-27T07:04:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 --2- 172.21.15.12:0/271096089 >> v2:172.21.15.53:3300/0 conn(0x7fef7806e8e0 0x7fef781482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 --2- 172.21.15.12:0/271096089 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 0x7fef78147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 --2- 172.21.15.12:0/271096089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 0x7fef780738a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 -- 172.21.15.12:0/271096089 >> 172.21.15.12:0/271096089 conn(0x7fef780bb4f0 msgr2=0x7fef780bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 -- 172.21.15.12:0/271096089 shutdown_connections 2024-04-27T07:04:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 -- 172.21.15.12:0/271096089 wait complete. 2024-04-27T07:04:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 Processor -- start 2024-04-27T07:04:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.152+0000 7fef7d3c2640 1 -- start start 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef7d3c2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 0x7fef78143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef7d3c2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 0x7fef781e4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef7d3c2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef7806e8e0 0x7fef78142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef7d3c2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef7814bdc0 con 0x7fef7806e500 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef7d3c2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef7814bc40 con 0x7fef7806e8e0 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef7d3c2640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fef7814bf40 con 0x7fef7806cc90 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef767fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 0x7fef781e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef767fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 0x7fef781e4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46736/0 (socket says 172.21.15.12:46736) 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef767fc640 1 -- 172.21.15.12:0/4024836414 learned_addr learned my addr 172.21.15.12:0/4024836414 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef76ffd640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 0x7fef78143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef777fe640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef7806e8e0 0x7fef78142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef767fc640 1 -- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 msgr2=0x7fef78143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef767fc640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 0x7fef78143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef767fc640 1 -- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef7806e8e0 msgr2=0x7fef78142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef767fc640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef7806e8e0 0x7fef78142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef767fc640 1 -- 172.21.15.12:0/4024836414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef781e3b80 con 0x7fef7806e500 2024-04-27T07:04:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef76ffd640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 0x7fef78143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef767fc640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 0x7fef781e4050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fef6800d5f0 tx=0x7fef6800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef6f7fe640 1 -- 172.21.15.12:0/4024836414 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef680125f0 con 0x7fef7806e500 2024-04-27T07:04:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef6f7fe640 1 -- 172.21.15.12:0/4024836414 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68019070 con 0x7fef7806e500 2024-04-27T07:04:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef7d3c2640 1 -- 172.21.15.12:0/4024836414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef781e4590 con 0x7fef7806e500 2024-04-27T07:04:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef6f7fe640 1 -- 172.21.15.12:0/4024836414 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef680129b0 con 0x7fef7806e500 2024-04-27T07:04:48.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef7d3c2640 1 -- 172.21.15.12:0/4024836414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef781456d0 con 0x7fef7806e500 2024-04-27T07:04:48.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef7d3c2640 1 -- 172.21.15.12:0/4024836414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef78069330 con 0x7fef7806e500 2024-04-27T07:04:48.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef6f7fe640 1 -- 172.21.15.12:0/4024836414 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fef68012b50 con 0x7fef7806e500 2024-04-27T07:04:48.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef6f7fe640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef58068380 0x7fef5806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef6f7fe640 1 -- 172.21.15.12:0/4024836414 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fef68087780 con 0x7fef7806e500 2024-04-27T07:04:48.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.156+0000 7fef76ffd640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef58068380 0x7fef5806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.160+0000 7fef76ffd640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef58068380 0x7fef5806a840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef600af000 tx=0x7fef60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:48.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.160+0000 7fef6f7fe640 1 -- 172.21.15.12:0/4024836414 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fef68057eb0 con 0x7fef7806e500 2024-04-27T07:04:48.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.324+0000 7fef7d3c2640 1 -- 172.21.15.12:0/4024836414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef781465b0 con 0x7fef7806e500 2024-04-27T07:04:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef6f7fe640 1 -- 172.21.15.12:0/4024836414 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7fef680577b0 con 0x7fef7806e500 2024-04-27T07:04:48.333 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:48.333 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 -- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef58068380 msgr2=0x7fef5806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef58068380 0x7fef5806a840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef600af000 tx=0x7fef60008040 comp rx=0 tx=0).stop 2024-04-27T07:04:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 -- 172.21.15.12:0/4024836414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 msgr2=0x7fef781e4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 0x7fef781e4050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fef6800d5f0 tx=0x7fef6800dab0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 -- 172.21.15.12:0/4024836414 shutdown_connections 2024-04-27T07:04:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef58068380 0x7fef5806a840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef7806e8e0 0x7fef78142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef7806e500 0x7fef781e4050 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 --2- 172.21.15.12:0/4024836414 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef7806cc90 0x7fef78143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 -- 172.21.15.12:0/4024836414 >> 172.21.15.12:0/4024836414 conn(0x7fef780bb4f0 msgr2=0x7fef78148500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 -- 172.21.15.12:0/4024836414 shutdown_connections 2024-04-27T07:04:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.328+0000 7fef7d3c2640 1 -- 172.21.15.12:0/4024836414 wait complete. 2024-04-27T07:04:48.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:12.742740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:13.912898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:48.352 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f0090fd2640 1 Processor -- start 2024-04-27T07:04:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f0090fd2640 1 -- start start 2024-04-27T07:04:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f0090fd2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f008c06e370 0x7f008c13faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f0090fd2640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f008c0b4d70 con 0x7f008c06d300 2024-04-27T07:04:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f0090fd2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f008c073e60 con 0x7f008c0b3650 2024-04-27T07:04:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f0090fd2640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f008c0740d0 con 0x7f008c06e370 2024-04-27T07:04:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f008a59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f008c06e370 0x7f008c13faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f008ad9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f008c06d300 0x7f008c150420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49224/0 (socket says 172.21.15.12:49224) 2024-04-27T07:04:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f008ad9d640 1 -- 172.21.15.12:0/157883554 learned_addr learned my addr 172.21.15.12:0/157883554 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f008a59c640 1 -- 172.21.15.12:0/157883554 >> v1:172.21.15.53:6789/0 conn(0x7f008c0b3650 legacy=0x7f008c0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f008a59c640 1 -- 172.21.15.12:0/157883554 >> v1:172.21.15.12:6789/0 conn(0x7f008c06d300 legacy=0x7f008c150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f0089d9b640 1 -- 172.21.15.12:0/157883554 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3304228330 0 0) 0x7f008c0b4d70 con 0x7f008c06d300 2024-04-27T07:04:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.472+0000 7f008a59c640 1 -- 172.21.15.12:0/157883554 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008c0b2bf0 con 0x7f008c06e370 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008a59c640 1 --2- 172.21.15.12:0/157883554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c06e370 0x7f008c13faa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f007c0097c0 tx=0x7f007c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ddd8497fce7dee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0089d9b640 1 -- 172.21.15.12:0/157883554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c0b0070 con 0x7f008c06e370 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0089d9b640 1 -- 172.21.15.12:0/157883554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f007c0ab600 con 0x7f008c06e370 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0089d9b640 1 -- 172.21.15.12:0/157883554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c0ab9d0 con 0x7f008c06e370 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- 172.21.15.12:0/157883554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c06e370 msgr2=0x7f008c13faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 --2- 172.21.15.12:0/157883554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c06e370 0x7f008c13faa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f007c0097c0 tx=0x7f007c01c410 comp rx=0 tx=0).stop 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- 172.21.15.12:0/157883554 shutdown_connections 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 --2- 172.21.15.12:0/157883554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c06e370 0x7f008c13faa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- 172.21.15.12:0/157883554 >> 172.21.15.12:0/157883554 conn(0x7f008c0b93b0 msgr2=0x7f008c0b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- 172.21.15.12:0/157883554 shutdown_connections 2024-04-27T07:04:48.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- 172.21.15.12:0/157883554 wait complete. 2024-04-27T07:04:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 Processor -- start 2024-04-27T07:04:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- start start 2024-04-27T07:04:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f008c06d300 0x7f008c072190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f008c06e370 0x7f008c1deae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c0b3650 0x7f008c1df020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f008c1546f0 con 0x7f008c0b3650 2024-04-27T07:04:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f008c154570 con 0x7f008c06e370 2024-04-27T07:04:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f008c154870 con 0x7f008c06d300 2024-04-27T07:04:48.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c0b3650 0x7f008c1df020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c0b3650 0x7f008c1df020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46758/0 (socket says 172.21.15.12:46758) 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008ad9d640 1 -- 172.21.15.12:0/2005452540 learned_addr learned my addr 172.21.15.12:0/2005452540 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008a59c640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f008c06d300 0x7f008c072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008ad9d640 1 -- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f008c06d300 msgr2=0x7f008c072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008ad9d640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f008c06d300 0x7f008c072190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008ad9d640 1 -- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f008c06e370 msgr2=0x7f008c1deae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0083fff640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f008c06e370 0x7f008c1deae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008ad9d640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f008c06e370 0x7f008c1deae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008ad9d640 1 -- 172.21.15.12:0/2005452540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008c1467e0 con 0x7f008c0b3650 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f008ad9d640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c0b3650 0x7f008c1df020 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f008400c780 tx=0x7f008400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0082ffd640 1 -- 172.21.15.12:0/2005452540 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084018070 con 0x7f008c0b3650 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0082ffd640 1 -- 172.21.15.12:0/2005452540 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f008400a640 con 0x7f008c0b3650 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- 172.21.15.12:0/2005452540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008c072930 con 0x7f008c0b3650 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0082ffd640 1 -- 172.21.15.12:0/2005452540 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008400aa70 con 0x7f008c0b3650 2024-04-27T07:04:48.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.476+0000 7f0090fd2640 1 -- 172.21.15.12:0/2005452540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f008c071260 con 0x7f008c0b3650 2024-04-27T07:04:48.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.480+0000 7f0090fd2640 1 -- 172.21.15.12:0/2005452540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f008c071460 con 0x7f008c0b3650 2024-04-27T07:04:48.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.480+0000 7f0082ffd640 1 -- 172.21.15.12:0/2005452540 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f008401b070 con 0x7f008c0b3650 2024-04-27T07:04:48.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.480+0000 7f0082ffd640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f006c0683b0 0x7f006c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.480+0000 7f0082ffd640 1 -- 172.21.15.12:0/2005452540 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f00840861d0 con 0x7f008c0b3650 2024-04-27T07:04:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.480+0000 7f008a59c640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f006c0683b0 0x7f006c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.480+0000 7f008a59c640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f006c0683b0 0x7f006c06a870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f007c0ad000 tx=0x7f007c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:48.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.484+0000 7f0082ffd640 1 -- 172.21.15.12:0/2005452540 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f008c071460 con 0x7f008c0b3650 2024-04-27T07:04:48.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 -- 172.21.15.12:0/2005452540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f008c072b00 con 0x7f008c0b3650 2024-04-27T07:04:48.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0082ffd640 1 -- 172.21.15.12:0/2005452540 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0084056900 con 0x7f008c0b3650 2024-04-27T07:04:48.653 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:48.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 -- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f006c0683b0 msgr2=0x7f006c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f006c0683b0 0x7f006c06a870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f007c0ad000 tx=0x7f007c008040 comp rx=0 tx=0).stop 2024-04-27T07:04:48.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 -- 172.21.15.12:0/2005452540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c0b3650 msgr2=0x7f008c1df020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c0b3650 0x7f008c1df020 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f008400c780 tx=0x7f008400cc40 comp rx=0 tx=0).stop 2024-04-27T07:04:48.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 -- 172.21.15.12:0/2005452540 shutdown_connections 2024-04-27T07:04:48.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f006c0683b0 0x7f006c06a870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f008c0b3650 0x7f008c1df020 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f008c06e370 0x7f008c1deae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 --2- 172.21.15.12:0/2005452540 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f008c06d300 0x7f008c072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 -- 172.21.15.12:0/2005452540 >> 172.21.15.12:0/2005452540 conn(0x7f008c0b93b0 msgr2=0x7f008c13d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:48.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 -- 172.21.15.12:0/2005452540 shutdown_connections 2024-04-27T07:04:48.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.648+0000 7f0090fd2640 1 -- 172.21.15.12:0/2005452540 wait complete. 2024-04-27T07:04:48.673 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:04:48.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f96a0673640 1 Processor -- start 2024-04-27T07:04:48.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f96a0673640 1 -- start start 2024-04-27T07:04:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f96a0673640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96980b6480 0x7f96980b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f96a0673640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9698142010 0x7f96980b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f96a0673640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96980b6860 con 0x7f96981489c0 2024-04-27T07:04:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f96a0673640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f96980b84e0 con 0x7f9698142010 2024-04-27T07:04:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f96a0673640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f9698147ee0 con 0x7f96980b6480 2024-04-27T07:04:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969dc0e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f96981489c0 0x7f9698141900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49232/0 (socket says 172.21.15.12:49232) 2024-04-27T07:04:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969dc0e640 1 -- 172.21.15.12:0/2363152666 learned_addr learned my addr 172.21.15.12:0/2363152666 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969e40f640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96980b6480 0x7f96980b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969ec10640 1 --2- 172.21.15.12:0/2363152666 >> v2:172.21.15.53:3300/0 conn(0x7f9698142010 0x7f96980b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233941444 0 0) 0x7f96980b6860 con 0x7f96981489c0 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9684003530 con 0x7f96981489c0 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1870550264 0 0) 0x7f9684003530 con 0x7f96981489c0 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96980b6480 msgr2=0x7f96980b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96980b6480 0x7f96980b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 >> v2:172.21.15.53:3300/0 conn(0x7f9698142010 msgr2=0x7f96980b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 --2- 172.21.15.12:0/2363152666 >> v2:172.21.15.53:3300/0 conn(0x7f9698142010 0x7f96980b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96981481c0 con 0x7f96981489c0 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f9688001680 con 0x7f96981489c0 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969ec10640 1 --2- 172.21.15.12:0/2363152666 >> v2:172.21.15.53:3300/0 conn(0x7f9698142010 0x7f96980b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 >> v1:172.21.15.12:6789/0 conn(0x7f96981489c0 legacy=0x7f9698141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96980b6480 0x7f96840039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9684003f40 0x7f9684004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9684004d50 0x7f96840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9698147ee0 con 0x7f9684004d50 2024-04-27T07:04:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f968400ba80 con 0x7f9684003f40 2024-04-27T07:04:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f968400bc50 con 0x7f96980b6480 2024-04-27T07:04:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969ec10640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9684004d50 0x7f96840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969dc0e640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9684003f40 0x7f9684004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969e40f640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96980b6480 0x7f96840039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969ec10640 1 -- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96980b6480 msgr2=0x7f96840039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969ec10640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f96980b6480 0x7f96840039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969ec10640 1 -- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9684003f40 msgr2=0x7f9684004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969ec10640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9684003f40 0x7f9684004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.800+0000 7f969ec10640 1 -- 172.21.15.12:0/2363152666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9684003530 con 0x7f9684004d50 2024-04-27T07:04:48.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969ec10640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9684004d50 0x7f96840051d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9690007410 tx=0x7f969001cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9bd83b70b350a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:48.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96900b2070 con 0x7f9684004d50 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96900a5d30 con 0x7f9684004d50 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969d40d640 1 -- 172.21.15.12:0/2363152666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690009c80 con 0x7f9684004d50 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 -- 172.21.15.12:0/2363152666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9684004d50 msgr2=0x7f96840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 --2- 172.21.15.12:0/2363152666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9684004d50 0x7f96840051d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9690007410 tx=0x7f969001cf40 comp rx=0 tx=0).stop 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969ec10640 1 -- 172.21.15.12:0/2363152666 reap_dead start 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 -- 172.21.15.12:0/2363152666 shutdown_connections 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 -- 172.21.15.12:0/2363152666 >> 172.21.15.12:0/2363152666 conn(0x7f96980bd620 msgr2=0x7f96980bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 -- 172.21.15.12:0/2363152666 shutdown_connections 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 -- 172.21.15.12:0/2363152666 wait complete. 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 Processor -- start 2024-04-27T07:04:48.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 -- start start 2024-04-27T07:04:48.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9684004d50 0x7f96981476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96981cf310 0x7f9698139990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9698139ed0 0x7f96981e0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f96980b9ee0 con 0x7f96981cf310 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f96980b9d60 con 0x7f9698139ed0 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f96a0673640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f96980b5040 con 0x7f9684004d50 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969dc0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96981cf310 0x7f9698139990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969dc0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96981cf310 0x7f9698139990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46776/0 (socket says 172.21.15.12:46776) 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969dc0e640 1 -- 172.21.15.12:0/691845750 learned_addr learned my addr 172.21.15.12:0/691845750 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969e40f640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9684004d50 0x7f96981476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969ec10640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9698139ed0 0x7f96981e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969dc0e640 1 -- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9684004d50 msgr2=0x7f96981476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969dc0e640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9684004d50 0x7f96981476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969dc0e640 1 -- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9698139ed0 msgr2=0x7f96981e0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969dc0e640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9698139ed0 0x7f96981e0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969dc0e640 1 -- 172.21.15.12:0/691845750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969813a350 con 0x7f96981cf310 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.804+0000 7f969ec10640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9698139ed0 0x7f96981e0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f969dc0e640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96981cf310 0x7f9698139990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9688011980 tx=0x7f9688011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f968effd640 1 -- 172.21.15.12:0/691845750 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688010040 con 0x7f96981cf310 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f968effd640 1 -- 172.21.15.12:0/691845750 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968801a070 con 0x7f96981cf310 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f968effd640 1 -- 172.21.15.12:0/691845750 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968800fe20 con 0x7f96981cf310 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f96a0673640 1 -- 172.21.15.12:0/691845750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96981cf860 con 0x7f96981cf310 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f96a0673640 1 -- 172.21.15.12:0/691845750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96981439c0 con 0x7f96981cf310 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f96a0673640 1 -- 172.21.15.12:0/691845750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9698068c20 con 0x7f96981cf310 2024-04-27T07:04:48.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f968effd640 1 -- 172.21.15.12:0/691845750 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f968801f460 con 0x7f96981cf310 2024-04-27T07:04:48.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f968effd640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f967c068380 0x7f967c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:48.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f968effd640 1 -- 172.21.15.12:0/691845750 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9688089c80 con 0x7f96981cf310 2024-04-27T07:04:48.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f969e40f640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f967c068380 0x7f967c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:48.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.808+0000 7f969e40f640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f967c068380 0x7f967c06a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f969813af90 tx=0x7f9694008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:48.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.812+0000 7f968effd640 1 -- 172.21.15.12:0/691845750 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f968805a2d0 con 0x7f96981cf310 2024-04-27T07:04:48.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.976+0000 7f96a0673640 1 -- 172.21.15.12:0/691845750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f969806bb80 con 0x7f96981cf310 2024-04-27T07:04:48.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.976+0000 7f968effd640 1 -- 172.21.15.12:0/691845750 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f9688059bd0 con 0x7f96981cf310 2024-04-27T07:04:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 -- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f967c068380 msgr2=0x7f967c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f967c068380 0x7f967c06a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f969813af90 tx=0x7f9694008040 comp rx=0 tx=0).stop 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 -- 172.21.15.12:0/691845750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96981cf310 msgr2=0x7f9698139990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96981cf310 0x7f9698139990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9688011980 tx=0x7f9688011e40 comp rx=0 tx=0).stop 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 -- 172.21.15.12:0/691845750 shutdown_connections 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f967c068380 0x7f967c06a840 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9698139ed0 0x7f96981e0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96981cf310 0x7f9698139990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 --2- 172.21.15.12:0/691845750 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9684004d50 0x7f96981476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 -- 172.21.15.12:0/691845750 >> 172.21.15.12:0/691845750 conn(0x7f96980bd620 msgr2=0x7f96980b92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 -- 172.21.15.12:0/691845750 shutdown_connections 2024-04-27T07:04:48.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:48.980+0000 7f96a0673640 1 -- 172.21.15.12:0/691845750 wait complete. 2024-04-27T07:04:49.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:04:49.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152f519640 1 Processor -- start 2024-04-27T07:04:49.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152f519640 1 -- start start 2024-04-27T07:04:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152f519640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f152806e370 0x7f152813faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152f519640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15280b4d70 con 0x7f152806d300 2024-04-27T07:04:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152f519640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1528073e60 con 0x7f15280b3650 2024-04-27T07:04:49.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152f519640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f15280740d0 con 0x7f152806e370 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152dab6640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f152806d300 0x7f1528150420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49248/0 (socket says 172.21.15.12:49248) 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152dab6640 1 -- 172.21.15.12:0/1016729657 learned_addr learned my addr 172.21.15.12:0/1016729657 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152d2b5640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f152806e370 0x7f152813faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4261757321 0 0) 0x7f15280b4d70 con 0x7f152806d300 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1514003530 con 0x7f152806d300 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1745697520 0 0) 0x7f1514003530 con 0x7f152806d300 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f152806e370 msgr2=0x7f152813faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f152806e370 0x7f152813faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 >> v1:172.21.15.53:6789/0 conn(0x7f15280b3650 legacy=0x7f15280b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15280b2bf0 con 0x7f152806d300 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f1520001680 con 0x7f152806d300 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 >> v1:172.21.15.12:6789/0 conn(0x7f152806d300 legacy=0x7f1528150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f152806e370 0x7f15140039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1514003fc0 0x7f1514004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1514004d50 0x7f15140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15280740d0 con 0x7f1514003fc0 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f151400bbc0 con 0x7f1514004d50 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f151400bdc0 con 0x7f152806e370 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152cab4640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1514003fc0 0x7f1514004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152dab6640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1514004d50 0x7f15140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152d2b5640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f152806e370 0x7f15140039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152cab4640 1 -- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f152806e370 msgr2=0x7f15140039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152cab4640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f152806e370 0x7f15140039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152cab4640 1 -- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1514004d50 msgr2=0x7f15140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152cab4640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1514004d50 0x7f15140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.128+0000 7f152cab4640 1 -- 172.21.15.12:0/1016729657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1514003530 con 0x7f1514003fc0 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152cab4640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1514003fc0 0x7f1514004810 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1518001750 tx=0x7f151801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09cccc22a1d2804 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15180013a0 con 0x7f1514003fc0 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15180052f0 con 0x7f1514003fc0 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f151ffff640 1 -- 172.21.15.12:0/1016729657 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518003840 con 0x7f1514003fc0 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 -- 172.21.15.12:0/1016729657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1514003fc0 msgr2=0x7f1514004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 --2- 172.21.15.12:0/1016729657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1514003fc0 0x7f1514004810 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1518001750 tx=0x7f151801c3e0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152dab6640 1 -- 172.21.15.12:0/1016729657 reap_dead start 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 -- 172.21.15.12:0/1016729657 shutdown_connections 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 -- 172.21.15.12:0/1016729657 >> 172.21.15.12:0/1016729657 conn(0x7f15280b93b0 msgr2=0x7f15280b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 -- 172.21.15.12:0/1016729657 shutdown_connections 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 -- 172.21.15.12:0/1016729657 wait complete. 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 Processor -- start 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 -- start start 2024-04-27T07:04:49.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1514003fc0 0x7f15280b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f152809e030 0x7f152813e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152813e7c0 0x7f15281364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15280b0c40 con 0x7f152813e7c0 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15280b0ac0 con 0x7f152809e030 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f15280b0dc0 con 0x7f1514003fc0 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152dab6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152813e7c0 0x7f15281364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152dab6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152813e7c0 0x7f15281364c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46786/0 (socket says 172.21.15.12:46786) 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152dab6640 1 -- 172.21.15.12:0/563247864 learned_addr learned my addr 172.21.15.12:0/563247864 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152cab4640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f152809e030 0x7f152813e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152d2b5640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1514003fc0 0x7f15280b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152dab6640 1 -- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1514003fc0 msgr2=0x7f15280b8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152dab6640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1514003fc0 0x7f15280b8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152dab6640 1 -- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f152809e030 msgr2=0x7f152813e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152dab6640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f152809e030 0x7f152813e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152dab6640 1 -- 172.21.15.12:0/563247864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152813ec50 con 0x7f152813e7c0 2024-04-27T07:04:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152dab6640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152813e7c0 0x7f15281364c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1528146c30 tx=0x7f152000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f151dffb640 1 -- 172.21.15.12:0/563247864 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520019070 con 0x7f152813e7c0 2024-04-27T07:04:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f151dffb640 1 -- 172.21.15.12:0/563247864 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f152000c690 con 0x7f152813e7c0 2024-04-27T07:04:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f152f519640 1 -- 172.21.15.12:0/563247864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152813d6c0 con 0x7f152813e7c0 2024-04-27T07:04:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.132+0000 7f151dffb640 1 -- 172.21.15.12:0/563247864 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152000ca40 con 0x7f152813e7c0 2024-04-27T07:04:49.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.136+0000 7f152f519640 1 -- 172.21.15.12:0/563247864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152809d470 con 0x7f152813e7c0 2024-04-27T07:04:49.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.136+0000 7f151dffb640 1 -- 172.21.15.12:0/563247864 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f152001c070 con 0x7f152813e7c0 2024-04-27T07:04:49.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.136+0000 7f152f519640 1 -- 172.21.15.12:0/563247864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1528068c20 con 0x7f152813e7c0 2024-04-27T07:04:49.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.136+0000 7f151dffb640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f150c068380 0x7f150c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.136+0000 7f151dffb640 1 -- 172.21.15.12:0/563247864 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1520085da0 con 0x7f152813e7c0 2024-04-27T07:04:49.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.136+0000 7f152d2b5640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f150c068380 0x7f150c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.136+0000 7f152d2b5640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f150c068380 0x7f150c06a840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1524007520 tx=0x7f152400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:49.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.140+0000 7f151dffb640 1 -- 172.21.15.12:0/563247864 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1520055da0 con 0x7f152813e7c0 2024-04-27T07:04:49.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.296+0000 7f152f519640 1 -- 172.21.15.12:0/563247864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f15280b26d0 con 0x7f152813e7c0 2024-04-27T07:04:49.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.296+0000 7f151dffb640 1 -- 172.21.15.12:0/563247864 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f152005a0a0 con 0x7f152813e7c0 2024-04-27T07:04:49.303 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:04:49.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 -- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f150c068380 msgr2=0x7f150c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f150c068380 0x7f150c06a840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1524007520 tx=0x7f152400b040 comp rx=0 tx=0).stop 2024-04-27T07:04:49.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 -- 172.21.15.12:0/563247864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152813e7c0 msgr2=0x7f15281364c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152813e7c0 0x7f15281364c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1528146c30 tx=0x7f152000eb90 comp rx=0 tx=0).stop 2024-04-27T07:04:49.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 -- 172.21.15.12:0/563247864 shutdown_connections 2024-04-27T07:04:49.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f150c068380 0x7f150c06a840 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f152813e7c0 0x7f15281364c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f152809e030 0x7f152813e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 --2- 172.21.15.12:0/563247864 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1514003fc0 0x7f15280b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 -- 172.21.15.12:0/563247864 >> 172.21.15.12:0/563247864 conn(0x7f15280b93b0 msgr2=0x7f15280ae7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:49.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 -- 172.21.15.12:0/563247864 shutdown_connections 2024-04-27T07:04:49.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.300+0000 7f152f519640 1 -- 172.21.15.12:0/563247864 wait complete. 2024-04-27T07:04:49.320 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:04:49.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.436+0000 7f7fc1b4a640 1 Processor -- start 2024-04-27T07:04:49.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.436+0000 7f7fc1b4a640 1 -- start start 2024-04-27T07:04:49.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fc1b4a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7fbc0b60b0 0x7f7fbc070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fc1b4a640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7fbc06ddc0 0x7f7fbc147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fc1b4a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fbc147720 con 0x7f7fbc06e200 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fc1b4a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f7fbc071f40 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fc1b4a640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7fbc073160 con 0x7f7fbc06ddc0 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fbb7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7fbc0b60b0 0x7f7fbc070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fbbfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7fbc06e200 0x7f7fbc0b46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49258/0 (socket says 172.21.15.12:49258) 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fbbfff640 1 -- 172.21.15.12:0/3803827483 learned_addr learned my addr 172.21.15.12:0/3803827483 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fba7fc640 1 -- 172.21.15.12:0/3803827483 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3038138469 0 0) 0x7f7fb40014a0 con 0x7f7fbc06e200 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fbb7fe640 1 -- 172.21.15.12:0/3803827483 >> v2:172.21.15.53:3300/0 conn(0x7f7fbc06ddc0 msgr2=0x7f7fbc147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fbaffd640 1 --2- 172.21.15.12:0/3803827483 >> v2:172.21.15.53:3300/0 conn(0x7f7fbc06ddc0 0x7f7fbc147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fbb7fe640 1 --2- 172.21.15.12:0/3803827483 >> v2:172.21.15.53:3300/0 conn(0x7f7fbc06ddc0 0x7f7fbc147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fbb7fe640 1 -- 172.21.15.12:0/3803827483 >> v1:172.21.15.12:6789/0 conn(0x7f7fbc06e200 legacy=0x7f7fbc0b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fbb7fe640 1 -- 172.21.15.12:0/3803827483 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc0b6490 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fbaffd640 1 --2- 172.21.15.12:0/3803827483 >> v2:172.21.15.53:3300/0 conn(0x7f7fbc06ddc0 0x7f7fbc147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:49.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fbb7fe640 1 --2- 172.21.15.12:0/3803827483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 0x7f7fbc070d60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc06ee00 tx=0x7f7fac01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d696a6df2417e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:49.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fba7fc640 1 -- 172.21.15.12:0/3803827483 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0b0070 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fba7fc640 1 -- 172.21.15.12:0/3803827483 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac0a4e50 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.440+0000 7f7fba7fc640 1 -- 172.21.15.12:0/3803827483 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0ab820 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3803827483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 msgr2=0x7f7fbc070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 --2- 172.21.15.12:0/3803827483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 0x7f7fbc070d60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc06ee00 tx=0x7f7fac01c480 comp rx=0 tx=0).stop 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3803827483 shutdown_connections 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 --2- 172.21.15.12:0/3803827483 >> v2:172.21.15.53:3300/0 conn(0x7f7fbc06ddc0 0x7f7fbc147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 --2- 172.21.15.12:0/3803827483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 0x7f7fbc070d60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3803827483 >> 172.21.15.12:0/3803827483 conn(0x7f7fbc0b9e80 msgr2=0x7f7fbc0ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3803827483 shutdown_connections 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3803827483 wait complete. 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 Processor -- start 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- start start 2024-04-27T07:04:49.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7fbc06ddc0 0x7f7fbc13ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fbc06e200 0x7f7fbc152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 0x7f7fbc152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0b4f40 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0b4dc0 con 0x7f7fbc06e200 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc0b50c0 con 0x7f7fbc06ddc0 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 0x7f7fbc152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 0x7f7fbc152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46800/0 (socket says 172.21.15.12:46800) 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbbfff640 1 -- 172.21.15.12:0/3339076664 learned_addr learned my addr 172.21.15.12:0/3339076664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbb7fe640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7fbc06ddc0 0x7f7fbc13ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbaffd640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fbc06e200 0x7f7fbc152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbbfff640 1 -- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7fbc06ddc0 msgr2=0x7f7fbc13ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbbfff640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7fbc06ddc0 0x7f7fbc13ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbbfff640 1 -- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fbc06e200 msgr2=0x7f7fbc152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbbfff640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fbc06e200 0x7f7fbc152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbbfff640 1 -- 172.21.15.12:0/3339076664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc0ae050 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbb7fe640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7fbc06ddc0 0x7f7fbc13ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:49.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fbbfff640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 0x7f7fbc152c90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400c7f0 tx=0x7f7fb400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:49.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fa3fff640 1 -- 172.21.15.12:0/3339076664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4018070 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fa3fff640 1 -- 172.21.15.12:0/3339076664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb400a640 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fa3fff640 1 -- 172.21.15.12:0/3339076664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb400aa30 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3339076664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc153810 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.444+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3339076664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc151f50 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.448+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3339076664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fbc069330 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.448+0000 7f7fa3fff640 1 -- 172.21.15.12:0/3339076664 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7fb400abd0 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.448+0000 7f7fa3fff640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7fa8068380 0x7f7fa806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.448+0000 7f7fa3fff640 1 -- 172.21.15.12:0/3339076664 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7fb4086020 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.448+0000 7f7fbb7fe640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7fa8068380 0x7f7fa806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.448+0000 7f7fbb7fe640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7fa8068380 0x7f7fa806a840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0ad000 tx=0x7f7fac0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:49.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.452+0000 7f7fa3fff640 1 -- 172.21.15.12:0/3339076664 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7fbc147720 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.648+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3339076664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fbc0ae2c0 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fa3fff640 1 -- 172.21.15.12:0/3339076664 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3527 (secure 0 0 0) 0x7f7fb40565e0 con 0x7f7fbc0b60b0 2024-04-27T07:04:49.657 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:49.657 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1908224930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1908224930},{"type":"v1","addr":"172.21.15.53:6835","nonce":1908224930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:04:14.961460+0000","modified":"2024-04-27T07:04:17.982838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4242":{"gid":4242,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4173401303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4173401303},{"type":"v1","addr":"172.21.15.12:6833","nonce":4173401303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4242,"qdb_cluster":[4242]},"id":1}]} 2024-04-27T07:04:49.657 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-04-27T07:04:49.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7fa8068380 msgr2=0x7f7fa806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7fa8068380 0x7f7fa806a840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0ad000 tx=0x7f7fac0a6000 comp rx=0 tx=0).stop 2024-04-27T07:04:49.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3339076664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 msgr2=0x7f7fbc152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 0x7f7fbc152c90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400c7f0 tx=0x7f7fb400ccb0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3339076664 shutdown_connections 2024-04-27T07:04:49.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7fa8068380 0x7f7fa806a840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fbc0b60b0 0x7f7fbc152c90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7fbc06e200 0x7f7fbc152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 --2- 172.21.15.12:0/3339076664 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7fbc06ddc0 0x7f7fbc13ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3339076664 >> 172.21.15.12:0/3339076664 conn(0x7f7fbc0b9e80 msgr2=0x7f7fbc139f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:49.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3339076664 shutdown_connections 2024-04-27T07:04:49.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.652+0000 7f7fc1b4a640 1 -- 172.21.15.12:0/3339076664 wait complete. 2024-04-27T07:04:49.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:49.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.796+0000 7f685c9d5640 1 Processor -- start 2024-04-27T07:04:49.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.796+0000 7f685c9d5640 1 -- start start 2024-04-27T07:04:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685c9d5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f68580b9a10 0x7f68580739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685c9d5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68580b9df0 con 0x7f6858073ee0 2024-04-27T07:04:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685c9d5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68580b47e0 con 0x7f685806ddc0 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685c9d5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f68580b4590 con 0x7f68580b9a10 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f6856d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6858073ee0 0x7f68581525b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49274/0 (socket says 172.21.15.12:49274) 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f6856d9d640 1 -- 172.21.15.12:0/2973523016 learned_addr learned my addr 172.21.15.12:0/2973523016 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685659c640 1 --2- 172.21.15.12:0/2973523016 >> v2:172.21.15.53:3300/0 conn(0x7f68580b9a10 0x7f68580739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 430832187 0 0) 0x7f68580b9df0 con 0x7f6858073ee0 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6840003530 con 0x7f6858073ee0 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2794955989 0 0) 0x7f6840003530 con 0x7f6858073ee0 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 >> v2:172.21.15.53:3300/0 conn(0x7f68580b9a10 msgr2=0x7f68580739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 --2- 172.21.15.12:0/2973523016 >> v2:172.21.15.53:3300/0 conn(0x7f68580b9a10 0x7f68580739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 >> v1:172.21.15.53:6789/0 conn(0x7f685806ddc0 legacy=0x7f6858148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68580b4ac0 con 0x7f6858073ee0 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f6850001680 con 0x7f6858073ee0 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 >> v1:172.21.15.12:6789/0 conn(0x7f6858073ee0 legacy=0x7f68581525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68400039c0 0x7f6840003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6840004810 0x7f6840004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f68400052f0 0x7f684000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f68580b4590 con 0x7f68400039c0 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f684000c1c0 con 0x7f6840004810 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f684000c3c0 con 0x7f68400052f0 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685659c640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68400039c0 0x7f6840003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f6855d9b640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6840004810 0x7f6840004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f6856d9d640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f68400052f0 0x7f684000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685659c640 1 -- 172.21.15.12:0/2973523016 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f68400052f0 msgr2=0x7f684000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685659c640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f68400052f0 0x7f684000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685659c640 1 -- 172.21.15.12:0/2973523016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6840004810 msgr2=0x7f6840004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685659c640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6840004810 0x7f6840004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685659c640 1 -- 172.21.15.12:0/2973523016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6840003530 con 0x7f68400039c0 2024-04-27T07:04:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f6856d9d640 1 -- 172.21.15.12:0/2973523016 reap_dead start 2024-04-27T07:04:49.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.800+0000 7f685659c640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68400039c0 0x7f6840003de0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6848009f90 tx=0x7f684801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30c17efb1c1e2b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68480b2070 con 0x7f68400039c0 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68480a5a30 con 0x7f68400039c0 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685559a640 1 -- 172.21.15.12:0/2973523016 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848009c10 con 0x7f68400039c0 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- 172.21.15.12:0/2973523016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68400039c0 msgr2=0x7f6840003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68400039c0 0x7f6840003de0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6848009f90 tx=0x7f684801cc40 comp rx=0 tx=0).stop 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- 172.21.15.12:0/2973523016 shutdown_connections 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 --2- 172.21.15.12:0/2973523016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68400039c0 0x7f6840003de0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- 172.21.15.12:0/2973523016 >> 172.21.15.12:0/2973523016 conn(0x7f6858067d10 msgr2=0x7f6858068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- 172.21.15.12:0/2973523016 shutdown_connections 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- 172.21.15.12:0/2973523016 wait complete. 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 Processor -- start 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- start start 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f68400039c0 0x7f68580aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580b00d0 0x7f68580b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68581d6620 0x7f68580b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f685814b400 con 0x7f68581d6620 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f685814b280 con 0x7f68580b00d0 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f685814b580 con 0x7f68400039c0 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6856d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68581d6620 0x7f68580b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6856d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68581d6620 0x7f68580b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46820/0 (socket says 172.21.15.12:46820) 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6856d9d640 1 -- 172.21.15.12:0/2202019029 learned_addr learned my addr 172.21.15.12:0/2202019029 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685659c640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f68400039c0 0x7f68580aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6855d9b640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580b00d0 0x7f68580b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6856d9d640 1 -- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f68400039c0 msgr2=0x7f68580aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6856d9d640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f68400039c0 0x7f68580aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6856d9d640 1 -- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580b00d0 msgr2=0x7f68580b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6856d9d640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580b00d0 0x7f68580b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6856d9d640 1 -- 172.21.15.12:0/2202019029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68580b9120 con 0x7f68581d6620 2024-04-27T07:04:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6856d9d640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68581d6620 0x7f68580b7be0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6858145940 tx=0x7f685000e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6846ffd640 1 -- 172.21.15.12:0/2202019029 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850019070 con 0x7f68581d6620 2024-04-27T07:04:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6846ffd640 1 -- 172.21.15.12:0/2202019029 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6850010510 con 0x7f68581d6620 2024-04-27T07:04:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- 172.21.15.12:0/2202019029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68580ae470 con 0x7f68581d6620 2024-04-27T07:04:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f6846ffd640 1 -- 172.21.15.12:0/2202019029 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850010900 con 0x7f68581d6620 2024-04-27T07:04:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.804+0000 7f685c9d5640 1 -- 172.21.15.12:0/2202019029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68580aea30 con 0x7f68581d6620 2024-04-27T07:04:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.808+0000 7f685c9d5640 1 -- 172.21.15.12:0/2202019029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68580aec30 con 0x7f68581d6620 2024-04-27T07:04:49.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.808+0000 7f6846ffd640 1 -- 172.21.15.12:0/2202019029 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f685001c070 con 0x7f68581d6620 2024-04-27T07:04:49.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.808+0000 7f6846ffd640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6838068380 0x7f683806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:49.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.808+0000 7f6846ffd640 1 -- 172.21.15.12:0/2202019029 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6850085d70 con 0x7f68581d6620 2024-04-27T07:04:49.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.808+0000 7f685659c640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6838068380 0x7f683806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:49.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.808+0000 7f685659c640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6838068380 0x7f683806a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f68480af000 tx=0x7f68480a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:49.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.812+0000 7f6846ffd640 1 -- 172.21.15.12:0/2202019029 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f68580aec30 con 0x7f68581d6620 2024-04-27T07:04:49.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.968+0000 7f685c9d5640 1 -- 172.21.15.12:0/2202019029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68580b9df0 con 0x7f68581d6620 2024-04-27T07:04:49.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f6846ffd640 1 -- 172.21.15.12:0/2202019029 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f68500563c0 con 0x7f68581d6620 2024-04-27T07:04:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:49.977 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 -- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6838068380 msgr2=0x7f683806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6838068380 0x7f683806a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f68480af000 tx=0x7f68480a7040 comp rx=0 tx=0).stop 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 -- 172.21.15.12:0/2202019029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68581d6620 msgr2=0x7f68580b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68581d6620 0x7f68580b7be0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6858145940 tx=0x7f685000e890 comp rx=0 tx=0).stop 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 -- 172.21.15.12:0/2202019029 shutdown_connections 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6838068380 0x7f683806a840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68581d6620 0x7f68580b7be0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68580b00d0 0x7f68580b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 --2- 172.21.15.12:0/2202019029 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f68400039c0 0x7f68580aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 -- 172.21.15.12:0/2202019029 >> 172.21.15.12:0/2202019029 conn(0x7f6858067d10 msgr2=0x7f6858145c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 -- 172.21.15.12:0/2202019029 shutdown_connections 2024-04-27T07:04:49.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:49.972+0000 7f685c9d5640 1 -- 172.21.15.12:0/2202019029 wait complete. 2024-04-27T07:04:49.995 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-27T07:04:49.995 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:04:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.100+0000 7f199d8b1640 1 Processor -- start 2024-04-27T07:04:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.100+0000 7f199d8b1640 1 -- start start 2024-04-27T07:04:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f199d8b1640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f19980734c0 0x7f1998143e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f199d8b1640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1998072520 con 0x7f19980b0bb0 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f199d8b1640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1998067f50 con 0x7f1998067b70 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f199d8b1640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f19980b0760 con 0x7f19980734c0 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f19977fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f19980b0bb0 0x7f1998144340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49286/0 (socket says 172.21.15.12:49286) 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f19977fe640 1 -- 172.21.15.12:0/2524525026 learned_addr learned my addr 172.21.15.12:0/2524525026 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f19967fc640 1 --2- 172.21.15.12:0/2524525026 >> v2:172.21.15.53:3300/0 conn(0x7f19980734c0 0x7f1998143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 935398070 0 0) 0x7f1998072520 con 0x7f19980b0bb0 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1984005740 con 0x7f19980b0bb0 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2754071391 0 0) 0x7f1984005740 con 0x7f19980b0bb0 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 >> v2:172.21.15.53:3300/0 conn(0x7f19980734c0 msgr2=0x7f1998143e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 --2- 172.21.15.12:0/2524525026 >> v2:172.21.15.53:3300/0 conn(0x7f19980734c0 0x7f1998143e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 >> v1:172.21.15.53:6789/0 conn(0x7f1998067b70 legacy=0x7f19981480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1998071e80 con 0x7f19980b0bb0 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f1990001a00 con 0x7f19980b0bb0 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 >> v1:172.21.15.12:6789/0 conn(0x7f19980b0bb0 legacy=0x7f1998144340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1984005c20 0x7f1984006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1984006a20 0x7f1984006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1984007430 0x7f198400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19980b0760 con 0x7f1984007430 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f198400e380 con 0x7f1984006a20 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f198400e580 con 0x7f1984005c20 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f19977fe640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1984007430 0x7f198400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f19967fc640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1984006a20 0x7f1984006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1996ffd640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1984005c20 0x7f1984006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f19977fe640 1 -- 172.21.15.12:0/2524525026 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1984005c20 msgr2=0x7f1984006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f19977fe640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1984005c20 0x7f1984006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f19977fe640 1 -- 172.21.15.12:0/2524525026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1984006a20 msgr2=0x7f1984006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f19977fe640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1984006a20 0x7f1984006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f19977fe640 1 -- 172.21.15.12:0/2524525026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1984005740 con 0x7f1984007430 2024-04-27T07:04:50.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.104+0000 7f1996ffd640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1984005c20 0x7f1984006040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 -- 172.21.15.12:0/2524525026 reap_dead start 2024-04-27T07:04:50.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1984007430 0x7f198400dbe0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f199000a760 tx=0x7f199001f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb2085fc5c1a5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199000a3b0 con 0x7f1984007430 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19900aa410 con 0x7f1984007430 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f1995ffb640 1 -- 172.21.15.12:0/2524525026 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19900aa800 con 0x7f1984007430 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 -- 172.21.15.12:0/2524525026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1984007430 msgr2=0x7f198400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1984007430 0x7f198400dbe0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f199000a760 tx=0x7f199001f9e0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 -- 172.21.15.12:0/2524525026 shutdown_connections 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 --2- 172.21.15.12:0/2524525026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1984007430 0x7f198400dbe0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 -- 172.21.15.12:0/2524525026 >> 172.21.15.12:0/2524525026 conn(0x7f19980b9e80 msgr2=0x7f19980ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 -- 172.21.15.12:0/2524525026 shutdown_connections 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 -- 172.21.15.12:0/2524525026 wait complete. 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 Processor -- start 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 -- start start 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1984007430 0x7f1998122d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19981dd070 0x7f19981dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1998142040 0x7f19981424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1998072b50 con 0x7f1998142040 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f19980729d0 con 0x7f1984007430 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f199d8b1640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f1998072cd0 con 0x7f19981dd070 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1998142040 0x7f19981424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1998142040 0x7f19981424c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46840/0 (socket says 172.21.15.12:46840) 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 -- 172.21.15.12:0/1273223232 learned_addr learned my addr 172.21.15.12:0/1273223232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f1996ffd640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1984007430 0x7f1998122d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19967fc640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19981dd070 0x7f19981dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 -- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19981dd070 msgr2=0x7f19981dd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19981dd070 0x7f19981dd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 -- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1984007430 msgr2=0x7f1998122d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1984007430 0x7f1998122d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 -- 172.21.15.12:0/1273223232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19981dda30 con 0x7f1998142040 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f1996ffd640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1984007430 0x7f1998122d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:50.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f19977fe640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1998142040 0x7f19981424c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f19900ad040 tx=0x7f1984007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.108+0000 7f198ffff640 1 -- 172.21.15.12:0/1273223232 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19900bd070 con 0x7f1998142040 2024-04-27T07:04:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.112+0000 7f198ffff640 1 -- 172.21.15.12:0/1273223232 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1990001260 con 0x7f1998142040 2024-04-27T07:04:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.112+0000 7f198ffff640 1 -- 172.21.15.12:0/1273223232 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19900aa7c0 con 0x7f1998142040 2024-04-27T07:04:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.112+0000 7f199d8b1640 1 -- 172.21.15.12:0/1273223232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19981222d0 con 0x7f1998142040 2024-04-27T07:04:50.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.112+0000 7f199d8b1640 1 -- 172.21.15.12:0/1273223232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1998123f80 con 0x7f1998142040 2024-04-27T07:04:50.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.112+0000 7f198dffb640 1 -- 172.21.15.12:0/1273223232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19700052a0 con 0x7f1998142040 2024-04-27T07:04:50.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.112+0000 7f198ffff640 1 -- 172.21.15.12:0/1273223232 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f19900b3050 con 0x7f1998142040 2024-04-27T07:04:50.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.112+0000 7f198ffff640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1978068380 0x7f197806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.112+0000 7f198ffff640 1 -- 172.21.15.12:0/1273223232 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f19901236b0 con 0x7f1998142040 2024-04-27T07:04:50.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.112+0000 7f1996ffd640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1978068380 0x7f197806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.112+0000 7f1996ffd640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1978068380 0x7f197806a840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1980001680 tx=0x7f1980009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:50.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.116+0000 7f198ffff640 1 -- 172.21.15.12:0/1273223232 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f19900196f0 con 0x7f1998142040 2024-04-27T07:04:50.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.312+0000 7f198dffb640 1 -- 172.21.15.12:0/1273223232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1970005590 con 0x7f1998142040 2024-04-27T07:04:50.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.312+0000 7f198ffff640 1 -- 172.21.15.12:0/1273223232 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3527 (secure 0 0 0) 0x7f19900f3c60 con 0x7f1998142040 2024-04-27T07:04:50.320 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:50.321 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1908224930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1908224930},{"type":"v1","addr":"172.21.15.53:6835","nonce":1908224930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:04:14.961460+0000","modified":"2024-04-27T07:04:17.982838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4242":{"gid":4242,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4173401303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4173401303},{"type":"v1","addr":"172.21.15.12:6833","nonce":4173401303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4242,"qdb_cluster":[4242]},"id":1}]} 2024-04-27T07:04:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-04-27T07:04:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 -- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1978068380 msgr2=0x7f197806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1978068380 0x7f197806a840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1980001680 tx=0x7f1980009040 comp rx=0 tx=0).stop 2024-04-27T07:04:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 -- 172.21.15.12:0/1273223232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1998142040 msgr2=0x7f19981424c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1998142040 0x7f19981424c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f19900ad040 tx=0x7f1984007a50 comp rx=0 tx=0).stop 2024-04-27T07:04:50.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 -- 172.21.15.12:0/1273223232 shutdown_connections 2024-04-27T07:04:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1978068380 0x7f197806a840 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1998142040 0x7f19981424c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f19981dd070 0x7f19981dd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 --2- 172.21.15.12:0/1273223232 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1984007430 0x7f1998122d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 -- 172.21.15.12:0/1273223232 >> 172.21.15.12:0/1273223232 conn(0x7f19980b9e80 msgr2=0x7f19980ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 -- 172.21.15.12:0/1273223232 shutdown_connections 2024-04-27T07:04:50.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.316+0000 7f198dffb640 1 -- 172.21.15.12:0/1273223232 wait complete. 2024-04-27T07:04:50.338 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:04:50.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.444+0000 7f14b0257640 1 Processor -- start 2024-04-27T07:04:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.444+0000 7f14b0257640 1 -- start start 2024-04-27T07:04:50.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14b0257640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a806cc90 0x7f14a80b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14b0257640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a806ea90 con 0x7f14a806e6b0 2024-04-27T07:04:50.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14b0257640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a80b2320 con 0x7f14a8148180 2024-04-27T07:04:50.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14b0257640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f14a806dd00 con 0x7f14a806cc90 2024-04-27T07:04:50.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14adff3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f14a806e6b0 0x7f14a8141790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49302/0 (socket says 172.21.15.12:49302) 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14adff3640 1 -- 172.21.15.12:0/84666286 learned_addr learned my addr 172.21.15.12:0/84666286 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14ad7f2640 1 --2- 172.21.15.12:0/84666286 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a806cc90 0x7f14a80b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 500874598 0 0) 0x7f14a806ea90 con 0x7f14a806e6b0 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1494003530 con 0x7f14a806e6b0 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 33282815 0 0) 0x7f1494003530 con 0x7f14a806e6b0 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a806cc90 msgr2=0x7f14a80b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 --2- 172.21.15.12:0/84666286 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a806cc90 0x7f14a80b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 >> v1:172.21.15.53:6789/0 conn(0x7f14a8148180 legacy=0x7f14a81524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a80b2600 con 0x7f14a806e6b0 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f14a4001680 con 0x7f14a806e6b0 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 >> v1:172.21.15.12:6789/0 conn(0x7f14a806e6b0 legacy=0x7f14a8141790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 --2- 172.21.15.12:0/84666286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14940039f0 0x7f1494003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 --2- 172.21.15.12:0/84666286 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a806cc90 0x7f1494004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 --2- 172.21.15.12:0/84666286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1494004d50 0x7f14940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f149400bb60 con 0x7f14940039f0 2024-04-27T07:04:50.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f149400bce0 con 0x7f1494004d50 2024-04-27T07:04:50.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f149400bee0 con 0x7f14a806cc90 2024-04-27T07:04:50.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14adff3640 1 --2- 172.21.15.12:0/84666286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14940039f0 0x7f1494003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14adff3640 1 -- 172.21.15.12:0/84666286 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a806cc90 msgr2=0x7f1494004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:50.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14adff3640 1 --2- 172.21.15.12:0/84666286 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a806cc90 0x7f1494004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14adff3640 1 -- 172.21.15.12:0/84666286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1494004d50 msgr2=0x7f14940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:50.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14adff3640 1 --2- 172.21.15.12:0/84666286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1494004d50 0x7f14940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14adff3640 1 -- 172.21.15.12:0/84666286 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1494003530 con 0x7f14940039f0 2024-04-27T07:04:50.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14adff3640 1 --2- 172.21.15.12:0/84666286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14940039f0 0x7f1494003e10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14a400c110 tx=0x7f14a401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea057c667466122 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:50.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4007c30 con 0x7f14940039f0 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a40a9020 con 0x7f14940039f0 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.448+0000 7f14acff1640 1 -- 172.21.15.12:0/84666286 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4005b20 con 0x7f14940039f0 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- 172.21.15.12:0/84666286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14940039f0 msgr2=0x7f1494003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 --2- 172.21.15.12:0/84666286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14940039f0 0x7f1494003e10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14a400c110 tx=0x7f14a401cba0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14ae7f4640 1 -- 172.21.15.12:0/84666286 reap_dead start 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- 172.21.15.12:0/84666286 shutdown_connections 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- 172.21.15.12:0/84666286 >> 172.21.15.12:0/84666286 conn(0x7f14a80bb4f0 msgr2=0x7f14a80bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- 172.21.15.12:0/84666286 shutdown_connections 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- 172.21.15.12:0/84666286 wait complete. 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 Processor -- start 2024-04-27T07:04:50.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- start start 2024-04-27T07:04:50.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14a813d1c0 0x7f14a814c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14a814cb90 0x7f14a81e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a81e4f20 0x7f14a81e53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f14a8154610 con 0x7f14a813d1c0 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f14a806df30 con 0x7f14a814cb90 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f14a8154790 con 0x7f14a81e4f20 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14adff3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14a813d1c0 0x7f14a814c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14adff3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14a813d1c0 0x7f14a814c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46856/0 (socket says 172.21.15.12:46856) 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14adff3640 1 -- 172.21.15.12:0/1090437175 learned_addr learned my addr 172.21.15.12:0/1090437175 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14ae7f4640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a81e4f20 0x7f14a81e53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14ad7f2640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14a814cb90 0x7f14a81e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14adff3640 1 -- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a81e4f20 msgr2=0x7f14a81e53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14adff3640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a81e4f20 0x7f14a81e53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14adff3640 1 -- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14a814cb90 msgr2=0x7f14a81e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14adff3640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14a814cb90 0x7f14a81e49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14adff3640 1 -- 172.21.15.12:0/1090437175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a813d5a0 con 0x7f14a813d1c0 2024-04-27T07:04:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14ae7f4640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a81e4f20 0x7f14a81e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14adff3640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14a813d1c0 0x7f14a814c650 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14a401c9d0 tx=0x7f1494005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f149e7fc640 1 -- 172.21.15.12:0/1090437175 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4004cc0 con 0x7f14a813d1c0 2024-04-27T07:04:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f149e7fc640 1 -- 172.21.15.12:0/1090437175 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a40b2070 con 0x7f14a813d1c0 2024-04-27T07:04:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f149e7fc640 1 -- 172.21.15.12:0/1090437175 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4004e60 con 0x7f14a813d1c0 2024-04-27T07:04:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- 172.21.15.12:0/1090437175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a81505f0 con 0x7f14a813d1c0 2024-04-27T07:04:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.452+0000 7f14b0257640 1 -- 172.21.15.12:0/1090437175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a8146a60 con 0x7f14a813d1c0 2024-04-27T07:04:50.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.456+0000 7f14b0257640 1 -- 172.21.15.12:0/1090437175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a8069330 con 0x7f14a813d1c0 2024-04-27T07:04:50.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.456+0000 7f149e7fc640 1 -- 172.21.15.12:0/1090437175 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f14a40072f0 con 0x7f14a813d1c0 2024-04-27T07:04:50.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.456+0000 7f149e7fc640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f148c068380 0x7f148c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.456+0000 7f149e7fc640 1 -- 172.21.15.12:0/1090437175 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f14a411fe30 con 0x7f14a813d1c0 2024-04-27T07:04:50.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.456+0000 7f14ad7f2640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f148c068380 0x7f148c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.456+0000 7f14ad7f2640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f148c068380 0x7f148c06a840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f14a8152ae0 tx=0x7f1498008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:50.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.460+0000 7f149e7fc640 1 -- 172.21.15.12:0/1090437175 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14a40168b0 con 0x7f14a813d1c0 2024-04-27T07:04:50.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.652+0000 7f14b0257640 1 -- 172.21.15.12:0/1090437175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14a814c270 con 0x7f14a813d1c0 2024-04-27T07:04:50.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.652+0000 7f149e7fc640 1 -- 172.21.15.12:0/1090437175 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3527 (secure 0 0 0) 0x7f14a40f0480 con 0x7f14a813d1c0 2024-04-27T07:04:50.658 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:50.658 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/1908224930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":1908224930},{"type":"v1","addr":"172.21.15.53:6835","nonce":1908224930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:04:14.961460+0000","modified":"2024-04-27T07:04:17.982838+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4242":{"gid":4242,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4173401303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4173401303},{"type":"v1","addr":"172.21.15.12:6833","nonce":4173401303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4242,"qdb_cluster":[4242]},"id":1}]} 2024-04-27T07:04:50.658 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-04-27T07:04:50.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.652+0000 7f14b0257640 1 -- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f148c068380 msgr2=0x7f148c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.652+0000 7f14b0257640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f148c068380 0x7f148c06a840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f14a8152ae0 tx=0x7f1498008040 comp rx=0 tx=0).stop 2024-04-27T07:04:50.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.652+0000 7f14b0257640 1 -- 172.21.15.12:0/1090437175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14a813d1c0 msgr2=0x7f14a814c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.652+0000 7f14b0257640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14a813d1c0 0x7f14a814c650 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14a401c9d0 tx=0x7f1494005840 comp rx=0 tx=0).stop 2024-04-27T07:04:50.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.656+0000 7f14b0257640 1 -- 172.21.15.12:0/1090437175 shutdown_connections 2024-04-27T07:04:50.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.656+0000 7f14b0257640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f148c068380 0x7f148c06a840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.656+0000 7f14b0257640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f14a81e4f20 0x7f14a81e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.656+0000 7f14b0257640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14a814cb90 0x7f14a81e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.656+0000 7f14b0257640 1 --2- 172.21.15.12:0/1090437175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14a813d1c0 0x7f14a814c650 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.656+0000 7f14b0257640 1 -- 172.21.15.12:0/1090437175 >> 172.21.15.12:0/1090437175 conn(0x7f14a80bb4f0 msgr2=0x7f14a813db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:50.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.656+0000 7f14b0257640 1 -- 172.21.15.12:0/1090437175 shutdown_connections 2024-04-27T07:04:50.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.656+0000 7f14b0257640 1 -- 172.21.15.12:0/1090437175 wait complete. 2024-04-27T07:04:50.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:50.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.792+0000 7f08b5699640 1 Processor -- start 2024-04-27T07:04:50.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.792+0000 7f08b5699640 1 -- start start 2024-04-27T07:04:50.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08b5699640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08b006cc90 0x7f08b00b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08b5699640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b006ea90 con 0x7f08b006e6b0 2024-04-27T07:04:50.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08b5699640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b00b2320 con 0x7f08b0148180 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08b5699640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f08b006dd00 con 0x7f08b006cc90 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08aeffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f08b006e6b0 0x7f08b0141790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49308/0 (socket says 172.21.15.12:49308) 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08aeffd640 1 -- 172.21.15.12:0/1499477049 learned_addr learned my addr 172.21.15.12:0/1499477049 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 378656910 0 0) 0x7f08b006ea90 con 0x7f08b006e6b0 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0898003530 con 0x7f08b006e6b0 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08ae7fc640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08b006cc90 0x7f08b00b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3442969139 0 0) 0x7f0898003530 con 0x7f08b006e6b0 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08b006cc90 msgr2=0x7f08b00b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08b006cc90 0x7f08b00b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 >> v1:172.21.15.53:6789/0 conn(0x7f08b0148180 legacy=0x7f08b01524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b00b2600 con 0x7f08b006e6b0 2024-04-27T07:04:50.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f08a0001680 con 0x7f08b006e6b0 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 >> v1:172.21.15.12:6789/0 conn(0x7f08b006e6b0 legacy=0x7f08b0141790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08b006cc90 0x7f08980039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898004020 0x7f0898004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0898004d50 0x7f08980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f089800bbe0 con 0x7f0898004020 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f089800bd90 con 0x7f0898004d50 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f089800bf90 con 0x7f08b006cc90 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08ae7fc640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898004020 0x7f0898004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08af7fe640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0898004d50 0x7f08980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08aeffd640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08b006cc90 0x7f08980039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08ae7fc640 1 -- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08b006cc90 msgr2=0x7f08980039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:50.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08ae7fc640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f08b006cc90 0x7f08980039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08ae7fc640 1 -- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0898004d50 msgr2=0x7f08980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:50.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08ae7fc640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0898004d50 0x7f08980051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08ae7fc640 1 -- 172.21.15.12:0/1499477049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0898003530 con 0x7f0898004020 2024-04-27T07:04:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08ae7fc640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898004020 0x7f0898004810 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f08a400ba30 tx=0x7f08a401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb77edac6f72aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40b3070 con 0x7f0898004020 2024-04-27T07:04:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a40ae630 con 0x7f0898004020 2024-04-27T07:04:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.796+0000 7f08adffb640 1 -- 172.21.15.12:0/1499477049 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40aa050 con 0x7f0898004020 2024-04-27T07:04:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- 172.21.15.12:0/1499477049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898004020 msgr2=0x7f0898004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 --2- 172.21.15.12:0/1499477049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0898004020 0x7f0898004810 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f08a400ba30 tx=0x7f08a401c6a0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08af7fe640 1 -- 172.21.15.12:0/1499477049 reap_dead start 2024-04-27T07:04:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- 172.21.15.12:0/1499477049 shutdown_connections 2024-04-27T07:04:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- 172.21.15.12:0/1499477049 >> 172.21.15.12:0/1499477049 conn(0x7f08b00bb4f0 msgr2=0x7f08b00bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:50.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- 172.21.15.12:0/1499477049 shutdown_connections 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- 172.21.15.12:0/1499477049 wait complete. 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 Processor -- start 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- start start 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0898004020 0x7f08b01516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b014c570 0x7f08b014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08b01e41d0 0x7f08b01e4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0154610 con 0x7f08b014c570 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f08b006df30 con 0x7f08b01e41d0 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f08b0154790 con 0x7f0898004020 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08ae7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b014c570 0x7f08b014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08ae7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b014c570 0x7f08b014c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46870/0 (socket says 172.21.15.12:46870) 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08ae7fc640 1 -- 172.21.15.12:0/2577132199 learned_addr learned my addr 172.21.15.12:0/2577132199 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:50.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08aeffd640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0898004020 0x7f08b01516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08af7fe640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08b01e41d0 0x7f08b01e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08ae7fc640 1 -- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0898004020 msgr2=0x7f08b01516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08ae7fc640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0898004020 0x7f08b01516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08ae7fc640 1 -- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08b01e41d0 msgr2=0x7f08b01e4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08ae7fc640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08b01e41d0 0x7f08b01e4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08ae7fc640 1 -- 172.21.15.12:0/2577132199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b014cf30 con 0x7f08b014c570 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08af7fe640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08b01e41d0 0x7f08b01e4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08ae7fc640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b014c570 0x7f08b014c9f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f08a40b1040 tx=0x7f0898005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08977fe640 1 -- 172.21.15.12:0/2577132199 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40b3040 con 0x7f08b014c570 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08977fe640 1 -- 172.21.15.12:0/2577132199 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a4004220 con 0x7f08b014c570 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- 172.21.15.12:0/2577132199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b0153120 con 0x7f08b014c570 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08977fe640 1 -- 172.21.15.12:0/2577132199 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a400b600 con 0x7f08b014c570 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- 172.21.15.12:0/2577132199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b0150570 con 0x7f08b014c570 2024-04-27T07:04:50.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.800+0000 7f08b5699640 1 -- 172.21.15.12:0/2577132199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b0069330 con 0x7f08b014c570 2024-04-27T07:04:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.804+0000 7f08977fe640 1 -- 172.21.15.12:0/2577132199 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f08a40c1060 con 0x7f08b014c570 2024-04-27T07:04:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.804+0000 7f08977fe640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f089c0683b0 0x7f089c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.804+0000 7f08977fe640 1 -- 172.21.15.12:0/2577132199 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f08a41208e0 con 0x7f08b014c570 2024-04-27T07:04:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.804+0000 7f08aeffd640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f089c0683b0 0x7f089c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.804+0000 7f08aeffd640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f089c0683b0 0x7f089c06a870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08a0001ec0 tx=0x7f08a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:50.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.808+0000 7f08977fe640 1 -- 172.21.15.12:0/2577132199 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f08b014cf30 con 0x7f08b014c570 2024-04-27T07:04:50.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.964+0000 7f08b5699640 1 -- 172.21.15.12:0/2577132199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08b00b8960 con 0x7f08b014c570 2024-04-27T07:04:50.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.968+0000 7f08977fe640 1 -- 172.21.15.12:0/2577132199 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16545 (secure 0 0 0) 0x7f08a40163b0 con 0x7f08b014c570 2024-04-27T07:04:50.975 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:50.975 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:16.599035+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:12.742740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:13.912898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:50.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 -- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f089c0683b0 msgr2=0x7f089c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f089c0683b0 0x7f089c06a870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08a0001ec0 tx=0x7f08a0009040 comp rx=0 tx=0).stop 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 -- 172.21.15.12:0/2577132199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b014c570 msgr2=0x7f08b014c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b014c570 0x7f08b014c9f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f08a40b1040 tx=0x7f0898005840 comp rx=0 tx=0).stop 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 -- 172.21.15.12:0/2577132199 shutdown_connections 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f089c0683b0 0x7f089c06a870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08b01e41d0 0x7f08b01e4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b014c570 0x7f08b014c9f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 --2- 172.21.15.12:0/2577132199 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0898004020 0x7f08b01516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 -- 172.21.15.12:0/2577132199 >> 172.21.15.12:0/2577132199 conn(0x7f08b00bb4f0 msgr2=0x7f08b013ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 -- 172.21.15.12:0/2577132199 shutdown_connections 2024-04-27T07:04:50.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:50.972+0000 7f08b5699640 1 -- 172.21.15.12:0/2577132199 wait complete. 2024-04-27T07:04:50.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-27T07:04:51.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1eebd58640 1 Processor -- start 2024-04-27T07:04:51.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1eebd58640 1 -- start start 2024-04-27T07:04:51.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1eebd58640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ee409acd0 0x7f1ee40b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1eebd58640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1ee41432d0 0x7f1ee4145e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1eebd58640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ee409b0b0 con 0x7f1ee409c6a0 2024-04-27T07:04:51.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1eebd58640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1ee409cc50 con 0x7f1ee409acd0 2024-04-27T07:04:51.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1eebd58640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee409cf30 con 0x7f1ee41432d0 2024-04-27T07:04:51.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1ee9af4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1ee409acd0 0x7f1ee40b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:51.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1eea2f5640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1ee41432d0 0x7f1ee4145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:51.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1ee92f3640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f1ee409c6a0 0x7f1ee4142bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:51080/0 (socket says 172.21.15.12:51080) 2024-04-27T07:04:51.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1ee92f3640 1 -- 172.21.15.12:0/1549475118 learned_addr learned my addr 172.21.15.12:0/1549475118 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:51.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1ee9af4640 1 -- 172.21.15.12:0/1549475118 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1ee41432d0 msgr2=0x7f1ee4145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:51.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1ee9af4640 1 --2- 172.21.15.12:0/1549475118 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1ee41432d0 0x7f1ee4145e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1ee9af4640 1 -- 172.21.15.12:0/1549475118 >> v1:172.21.15.53:6789/0 conn(0x7f1ee409c6a0 legacy=0x7f1ee4142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:51.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.112+0000 7f1ee9af4640 1 -- 172.21.15.12:0/1549475118 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee41463a0 con 0x7f1ee409acd0 2024-04-27T07:04:51.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1ee9af4640 1 --2- 172.21.15.12:0/1549475118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee409acd0 0x7f1ee40b69f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00097c0 tx=0x7f1ee001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bdba4a4b2bfcdfd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:51.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1ee8af2640 1 -- 172.21.15.12:0/1549475118 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee00b0070 con 0x7f1ee409acd0 2024-04-27T07:04:51.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1ee8af2640 1 -- 172.21.15.12:0/1549475118 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee00ab600 con 0x7f1ee409acd0 2024-04-27T07:04:51.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1ee8af2640 1 -- 172.21.15.12:0/1549475118 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee00ab9d0 con 0x7f1ee409acd0 2024-04-27T07:04:51.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1eebd58640 1 -- 172.21.15.12:0/1549475118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee409acd0 msgr2=0x7f1ee40b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:51.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1eebd58640 1 --2- 172.21.15.12:0/1549475118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee409acd0 0x7f1ee40b69f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00097c0 tx=0x7f1ee001c410 comp rx=0 tx=0).stop 2024-04-27T07:04:51.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1eebd58640 1 -- 172.21.15.12:0/1549475118 shutdown_connections 2024-04-27T07:04:51.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1eebd58640 1 --2- 172.21.15.12:0/1549475118 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1ee41432d0 0x7f1ee4145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1eebd58640 1 --2- 172.21.15.12:0/1549475118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee409acd0 0x7f1ee40b69f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1eebd58640 1 -- 172.21.15.12:0/1549475118 >> 172.21.15.12:0/1549475118 conn(0x7f1ee40b7060 msgr2=0x7f1ee40b7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:51.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1eebd58640 1 -- 172.21.15.12:0/1549475118 shutdown_connections 2024-04-27T07:04:51.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.116+0000 7f1eebd58640 1 -- 172.21.15.12:0/1549475118 wait complete. 2024-04-27T07:04:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 Processor -- start 2024-04-27T07:04:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 -- start start 2024-04-27T07:04:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1ee409acd0 0x7f1ee40afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee409c6a0 0x7f1ee40b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee41432d0 0x7f1ee40b9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4152f30 con 0x7f1ee41432d0 2024-04-27T07:04:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4152db0 con 0x7f1ee409c6a0 2024-04-27T07:04:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee41530b0 con 0x7f1ee409acd0 2024-04-27T07:04:51.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eea2f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee41432d0 0x7f1ee40b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eea2f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee41432d0 0x7f1ee40b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46884/0 (socket says 172.21.15.12:46884) 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eea2f5640 1 -- 172.21.15.12:0/2557873409 learned_addr learned my addr 172.21.15.12:0/2557873409 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1ee9af4640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1ee409acd0 0x7f1ee40afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1ee92f3640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee409c6a0 0x7f1ee40b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eea2f5640 1 -- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1ee409acd0 msgr2=0x7f1ee40afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eea2f5640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1ee409acd0 0x7f1ee40afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eea2f5640 1 -- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee409c6a0 msgr2=0x7f1ee40b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eea2f5640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee409c6a0 0x7f1ee40b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eea2f5640 1 -- 172.21.15.12:0/2557873409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee40b2920 con 0x7f1ee41432d0 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eea2f5640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee41432d0 0x7f1ee40b9760 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00efc0 tx=0x7f1edc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eda7fc640 1 -- 172.21.15.12:0/2557873409 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc018020 con 0x7f1ee41432d0 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eda7fc640 1 -- 172.21.15.12:0/2557873409 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1edc014480 con 0x7f1ee41432d0 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 -- 172.21.15.12:0/2557873409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee40b2c00 con 0x7f1ee41432d0 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eda7fc640 1 -- 172.21.15.12:0/2557873409 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc009e60 con 0x7f1ee41432d0 2024-04-27T07:04:51.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 -- 172.21.15.12:0/2557873409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee40b0560 con 0x7f1ee41432d0 2024-04-27T07:04:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eebd58640 1 -- 172.21.15.12:0/2557873409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee40b0760 con 0x7f1ee41432d0 2024-04-27T07:04:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eda7fc640 1 -- 172.21.15.12:0/2557873409 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1edc01b020 con 0x7f1ee41432d0 2024-04-27T07:04:51.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eda7fc640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ec80683b0 0x7f1ec806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.120+0000 7f1eda7fc640 1 -- 172.21.15.12:0/2557873409 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1edc087c10 con 0x7f1ee41432d0 2024-04-27T07:04:51.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.124+0000 7f1ee9af4640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ec80683b0 0x7f1ec806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:51.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.124+0000 7f1ee9af4640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ec80683b0 0x7f1ec806a870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00ad000 tx=0x7f1ee0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:51.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.124+0000 7f1eda7fc640 1 -- 172.21.15.12:0/2557873409 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ee40b0760 con 0x7f1ee41432d0 2024-04-27T07:04:51.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.316+0000 7f1eebd58640 1 -- 172.21.15.12:0/2557873409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1ee4068c20 con 0x7f1ee41432d0 2024-04-27T07:04:51.324 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-04-27T07:04:51.320+0000 7efe6513c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T07:04:51.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.320+0000 7f1eda7fc640 1 -- 172.21.15.12:0/2557873409 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f1edc022030 con 0x7f1ee41432d0 2024-04-27T07:04:51.326 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-27T07:04:51.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 -- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ec80683b0 msgr2=0x7f1ec806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:51.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ec80683b0 0x7f1ec806a870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00ad000 tx=0x7f1ee0008040 comp rx=0 tx=0).stop 2024-04-27T07:04:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 -- 172.21.15.12:0/2557873409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee41432d0 msgr2=0x7f1ee40b9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee41432d0 0x7f1ee40b9760 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00efc0 tx=0x7f1edc00c490 comp rx=0 tx=0).stop 2024-04-27T07:04:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 -- 172.21.15.12:0/2557873409 shutdown_connections 2024-04-27T07:04:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1ec80683b0 0x7f1ec806a870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ee41432d0 0x7f1ee40b9760 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ee409c6a0 0x7f1ee40b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 --2- 172.21.15.12:0/2557873409 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1ee409acd0 0x7f1ee40afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 -- 172.21.15.12:0/2557873409 >> 172.21.15.12:0/2557873409 conn(0x7f1ee40b7060 msgr2=0x7f1ee40b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 -- 172.21.15.12:0/2557873409 shutdown_connections 2024-04-27T07:04:51.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.324+0000 7f1eebd58640 1 -- 172.21.15.12:0/2557873409 wait complete. 2024-04-27T07:04:51.335 INFO:tasks.ceph.mds.a.smithi012.stderr: -1375> 2024-04-27T07:04:18.537+0000 7fd8e37d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-27T07:04:51.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-27T07:04:51.373 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2024-04-27T07:04:51.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f689520c640 1 Processor -- start 2024-04-27T07:04:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f689520c640 1 -- start start 2024-04-27T07:04:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f689520c640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f68900b4ae0 0x7f68901454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f689520c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68900b3db0 0x7f689009da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f689520c640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68901459f0 con 0x7f689006e410 2024-04-27T07:04:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f689520c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f6890144a80 con 0x7f68900b3db0 2024-04-27T07:04:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f689520c640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f68900b4ec0 con 0x7f68900b4ae0 2024-04-27T07:04:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f688e59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68900b3db0 0x7f689009da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f688e59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68900b3db0 0x7f689009da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46900/0 (socket says 172.21.15.12:46900) 2024-04-27T07:04:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f688e59c640 1 -- 172.21.15.12:0/901561100 learned_addr learned my addr 172.21.15.12:0/901561100 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f688e59c640 1 -- 172.21.15.12:0/901561100 >> v2:172.21.15.53:3300/0 conn(0x7f68900b4ae0 msgr2=0x7f68901454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f688e59c640 1 --2- 172.21.15.12:0/901561100 >> v2:172.21.15.53:3300/0 conn(0x7f68900b4ae0 0x7f68901454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f688e59c640 1 -- 172.21.15.12:0/901561100 >> v1:172.21.15.53:6789/0 conn(0x7f689006e410 legacy=0x7f6890147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.456+0000 7f688e59c640 1 -- 172.21.15.12:0/901561100 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6890144cb0 con 0x7f68900b3db0 2024-04-27T07:04:51.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688e59c640 1 --2- 172.21.15.12:0/901561100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 0x7f689009da80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f689009c170 tx=0x7f687801ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76282471b242a066 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:51.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688dd9b640 1 -- 172.21.15.12:0/901561100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780b4070 con 0x7f68900b3db0 2024-04-27T07:04:51.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688dd9b640 1 -- 172.21.15.12:0/901561100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68780a7b90 con 0x7f68900b3db0 2024-04-27T07:04:51.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688dd9b640 1 -- 172.21.15.12:0/901561100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780af440 con 0x7f68900b3db0 2024-04-27T07:04:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- 172.21.15.12:0/901561100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 msgr2=0x7f689009da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 --2- 172.21.15.12:0/901561100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 0x7f689009da80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f689009c170 tx=0x7f687801ebb0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- 172.21.15.12:0/901561100 shutdown_connections 2024-04-27T07:04:51.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 --2- 172.21.15.12:0/901561100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 0x7f689009da80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 --2- 172.21.15.12:0/901561100 >> v2:172.21.15.53:3300/0 conn(0x7f68900b4ae0 0x7f68901454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- 172.21.15.12:0/901561100 >> 172.21.15.12:0/901561100 conn(0x7f68900b93b0 msgr2=0x7f68900b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- 172.21.15.12:0/901561100 shutdown_connections 2024-04-27T07:04:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- 172.21.15.12:0/901561100 wait complete. 2024-04-27T07:04:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 Processor -- start 2024-04-27T07:04:51.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- start start 2024-04-27T07:04:51.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f689006e410 0x7f689013e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 0x7f68901dfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68900b4ae0 0x7f6890145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f689006d480 con 0x7f68900b3db0 2024-04-27T07:04:51.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f689006d300 con 0x7f68900b4ae0 2024-04-27T07:04:51.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f689006d600 con 0x7f689006e410 2024-04-27T07:04:51.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 0x7f68901dfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:51.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 0x7f68901dfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46910/0 (socket says 172.21.15.12:46910) 2024-04-27T07:04:51.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688e59c640 1 -- 172.21.15.12:0/2003168308 learned_addr learned my addr 172.21.15.12:0/2003168308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688e59c640 1 -- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f689006e410 msgr2=0x7f689013e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688e59c640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f689006e410 0x7f689013e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688e59c640 1 -- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68900b4ae0 msgr2=0x7f6890145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688f59e640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68900b4ae0 0x7f6890145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688e59c640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68900b4ae0 0x7f6890145e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688e59c640 1 -- 172.21.15.12:0/2003168308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68901e0040 con 0x7f68900b3db0 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f688e59c640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 0x7f68901dfb00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f687800b2c0 tx=0x7f687801b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f68877fe640 1 -- 172.21.15.12:0/2003168308 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780b4040 con 0x7f68900b3db0 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f68877fe640 1 -- 172.21.15.12:0/2003168308 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68780c1070 con 0x7f68900b3db0 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- 172.21.15.12:0/2003168308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68901e02f0 con 0x7f68900b3db0 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f68877fe640 1 -- 172.21.15.12:0/2003168308 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780b9460 con 0x7f68900b3db0 2024-04-27T07:04:51.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.464+0000 7f689520c640 1 -- 172.21.15.12:0/2003168308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68901e08b0 con 0x7f68900b3db0 2024-04-27T07:04:51.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.468+0000 7f689520c640 1 -- 172.21.15.12:0/2003168308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6890068c20 con 0x7f68900b3db0 2024-04-27T07:04:51.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.468+0000 7f68877fe640 1 -- 172.21.15.12:0/2003168308 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f68780b7070 con 0x7f68900b3db0 2024-04-27T07:04:51.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.468+0000 7f68877fe640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6870068380 0x7f687006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:51.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.468+0000 7f68877fe640 1 -- 172.21.15.12:0/2003168308 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f6878123960 con 0x7f68900b3db0 2024-04-27T07:04:51.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.468+0000 7f688ed9d640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6870068380 0x7f687006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:51.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.468+0000 7f688ed9d640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6870068380 0x7f687006a840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6880009690 tx=0x7f6880006c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:51.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.472+0000 7f68877fe640 1 -- 172.21.15.12:0/2003168308 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f68780187a0 con 0x7f68900b3db0 2024-04-27T07:04:51.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:51.660+0000 7f689520c640 1 -- 172.21.15.12:0/2003168308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f68900b6d80 con 0x7f68900b3db0 2024-04-27T07:04:52.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.324+0000 7f68877fe640 1 -- 172.21.15.12:0/2003168308 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f68780f3ec0 con 0x7f68900b3db0 2024-04-27T07:04:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 -- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6870068380 msgr2=0x7f687006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6870068380 0x7f687006a840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6880009690 tx=0x7f6880006c00 comp rx=0 tx=0).stop 2024-04-27T07:04:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 -- 172.21.15.12:0/2003168308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 msgr2=0x7f68901dfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 0x7f68901dfb00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f687800b2c0 tx=0x7f687801b950 comp rx=0 tx=0).stop 2024-04-27T07:04:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 -- 172.21.15.12:0/2003168308 shutdown_connections 2024-04-27T07:04:52.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6870068380 0x7f687006a840 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f68900b4ae0 0x7f6890145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68900b3db0 0x7f68901dfb00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 --2- 172.21.15.12:0/2003168308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f689006e410 0x7f689013e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 -- 172.21.15.12:0/2003168308 >> 172.21.15.12:0/2003168308 conn(0x7f68900b93b0 msgr2=0x7f689006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 -- 172.21.15.12:0/2003168308 shutdown_connections 2024-04-27T07:04:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.328+0000 7f689520c640 1 -- 172.21.15.12:0/2003168308 wait complete. 2024-04-27T07:04:52.356 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-27T07:04:52.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcaee863640 1 Processor -- start 2024-04-27T07:04:52.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcaee863640 1 -- start start 2024-04-27T07:04:52.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcaee863640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae80b51b0 0x7fcae809d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:52.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcaee863640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fcae809dab0 0x7fcae8145030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:52.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcaee863640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae80b5590 con 0x7fcae80af2c0 2024-04-27T07:04:52.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcaee863640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fcae809e820 con 0x7fcae809dab0 2024-04-27T07:04:52.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcaee863640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fcae80afc40 con 0x7fcae80b51b0 2024-04-27T07:04:52.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcaece00640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcae80af2c0 0x7fcae8145570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49332/0 (socket says 172.21.15.12:49332) 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcaece00640 1 -- 172.21.15.12:0/1626722783 learned_addr learned my addr 172.21.15.12:0/1626722783 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcae7fff640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae80b51b0 0x7fcae809d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcae77fe640 1 --2- 172.21.15.12:0/1626722783 >> v2:172.21.15.53:3300/0 conn(0x7fcae809dab0 0x7fcae8145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 412684530 0 0) 0x7fcae80b5590 con 0x7fcae80af2c0 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.528+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcad0003530 con 0x7fcae80af2c0 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3822346166 0 0) 0x7fcad0003530 con 0x7fcae80af2c0 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae80b51b0 msgr2=0x7fcae809d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae80b51b0 0x7fcae809d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 >> v2:172.21.15.53:3300/0 conn(0x7fcae809dab0 msgr2=0x7fcae8145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 --2- 172.21.15.12:0/1626722783 >> v2:172.21.15.53:3300/0 conn(0x7fcae809dab0 0x7fcae8145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae809e440 con 0x7fcae80af2c0 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fcae0001a00 con 0x7fcae80af2c0 2024-04-27T07:04:52.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 >> v1:172.21.15.12:6789/0 conn(0x7fcae80af2c0 legacy=0x7fcae8145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcad00039c0 0x7fcad0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 0x7fcad0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae80b51b0 0x7fcad00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcae809e820 con 0x7fcad0004810 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcae80afc40 con 0x7fcad00039c0 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fcad000b9d0 con 0x7fcae80b51b0 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae77fe640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 0x7fcad0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae7fff640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcad00039c0 0x7fcad0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae77fe640 1 -- 172.21.15.12:0/1626722783 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae80b51b0 msgr2=0x7fcad00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae77fe640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae80b51b0 0x7fcad00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae77fe640 1 -- 172.21.15.12:0/1626722783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcad00039c0 msgr2=0x7fcad0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae77fe640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcad00039c0 0x7fcad0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:52.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae77fe640 1 -- 172.21.15.12:0/1626722783 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad0003530 con 0x7fcad0004810 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae77fe640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 0x7fcad0004c90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009ae0 tx=0x7fcadc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01ea3513670fdf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0096b0 con 0x7fcad0004810 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcadc0a9470 con 0x7fcad0004810 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcae6ffd640 1 -- 172.21.15.12:0/1626722783 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0a9860 con 0x7fcad0004810 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcaee863640 1 -- 172.21.15.12:0/1626722783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 msgr2=0x7fcad0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcaee863640 1 --2- 172.21.15.12:0/1626722783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 0x7fcad0004c90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009ae0 tx=0x7fcadc01ca70 comp rx=0 tx=0).stop 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcaece00640 1 -- 172.21.15.12:0/1626722783 reap_dead start 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcaee863640 1 -- 172.21.15.12:0/1626722783 shutdown_connections 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcaee863640 1 -- 172.21.15.12:0/1626722783 >> 172.21.15.12:0/1626722783 conn(0x7fcae80b9840 msgr2=0x7fcae80b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcaee863640 1 -- 172.21.15.12:0/1626722783 shutdown_connections 2024-04-27T07:04:52.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.532+0000 7fcaee863640 1 -- 172.21.15.12:0/1626722783 wait complete. 2024-04-27T07:04:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 Processor -- start 2024-04-27T07:04:52.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 -- start start 2024-04-27T07:04:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 0x7fcae8070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae80b2ee0 0x7fcae81df1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae81467f0 0x7fcae8070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcae814ffb0 con 0x7fcad0004810 2024-04-27T07:04:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcae814fe30 con 0x7fcae80b2ee0 2024-04-27T07:04:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fcae8150130 con 0x7fcae81467f0 2024-04-27T07:04:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 0x7fcae8070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 0x7fcae8070510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46934/0 (socket says 172.21.15.12:46934) 2024-04-27T07:04:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae7fff640 1 -- 172.21.15.12:0/127787046 learned_addr learned my addr 172.21.15.12:0/127787046 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:52.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae77fe640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae80b2ee0 0x7fcae81df1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaece00640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae81467f0 0x7fcae8070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae7fff640 1 -- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae81467f0 msgr2=0x7fcae8070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae7fff640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae81467f0 0x7fcae8070b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae7fff640 1 -- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae80b2ee0 msgr2=0x7fcae81df1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae7fff640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae80b2ee0 0x7fcae81df1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae7fff640 1 -- 172.21.15.12:0/127787046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae81460f0 con 0x7fcad0004810 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae77fe640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae80b2ee0 0x7fcae81df1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaece00640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae81467f0 0x7fcae8070b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae7fff640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 0x7fcae8070510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcad8011610 tx=0x7fcad8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae4ff9640 1 -- 172.21.15.12:0/127787046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad800f8f0 con 0x7fcad0004810 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae4ff9640 1 -- 172.21.15.12:0/127787046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad800fa90 con 0x7fcad0004810 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcae4ff9640 1 -- 172.21.15.12:0/127787046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad800fe40 con 0x7fcad0004810 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 -- 172.21.15.12:0/127787046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae8147000 con 0x7fcad0004810 2024-04-27T07:04:52.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 -- 172.21.15.12:0/127787046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae81de9f0 con 0x7fcad0004810 2024-04-27T07:04:52.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.536+0000 7fcaee863640 1 -- 172.21.15.12:0/127787046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae8068d00 con 0x7fcad0004810 2024-04-27T07:04:52.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.540+0000 7fcae4ff9640 1 -- 172.21.15.12:0/127787046 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fcad8015460 con 0x7fcad0004810 2024-04-27T07:04:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.540+0000 7fcae4ff9640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 0x7fcad406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.540+0000 7fcae4ff9640 1 -- 172.21.15.12:0/127787046 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7fcad8087230 con 0x7fcad0004810 2024-04-27T07:04:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.540+0000 7fcae77fe640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 0x7fcad406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:52.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.540+0000 7fcae77fe640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 0x7fcad406a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0ae000 tx=0x7fcadc0b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:52.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.544+0000 7fcae4ff9640 1 -- 172.21.15.12:0/127787046 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcad8019030 con 0x7fcad0004810 2024-04-27T07:04:52.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:52.704+0000 7fcaee863640 1 -- 172.21.15.12:0/127787046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcae80b5590 con 0x7fcad0004810 2024-04-27T07:04:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.332+0000 7fcae4ff9640 1 -- 172.21.15.12:0/127787046 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) ==== 181+0+0 (secure 0 0 0) 0x7fcad8057820 con 0x7fcad0004810 2024-04-27T07:04:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2024-04-27T07:04:53.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 -- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 msgr2=0x7fcad406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:53.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 0x7fcad406a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0ae000 tx=0x7fcadc0b3040 comp rx=0 tx=0).stop 2024-04-27T07:04:53.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 -- 172.21.15.12:0/127787046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 msgr2=0x7fcae8070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:53.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 0x7fcae8070510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcad8011610 tx=0x7fcad8011ad0 comp rx=0 tx=0).stop 2024-04-27T07:04:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 -- 172.21.15.12:0/127787046 shutdown_connections 2024-04-27T07:04:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fcad4068380 0x7fcad406a840 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fcae81467f0 0x7fcae8070b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcae80b2ee0 0x7fcae81df1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 --2- 172.21.15.12:0/127787046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcad0004810 0x7fcae8070510 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 -- 172.21.15.12:0/127787046 >> 172.21.15.12:0/127787046 conn(0x7fcae80b9840 msgr2=0x7fcae80b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 -- 172.21.15.12:0/127787046 shutdown_connections 2024-04-27T07:04:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.336+0000 7fcaee863640 1 -- 172.21.15.12:0/127787046 wait complete. 2024-04-27T07:04:53.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-27T07:04:53.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f0495c12640 1 Processor -- start 2024-04-27T07:04:53.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f0495c12640 1 -- start start 2024-04-27T07:04:53.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f0495c12640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04900b51b0 0x7f04900af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:53.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f0495c12640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f049009ce70 con 0x7f049009d570 2024-04-27T07:04:53.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f0495c12640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04900b5590 con 0x7f049009d9b0 2024-04-27T07:04:53.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f0495c12640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f049009e770 con 0x7f04900b51b0 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f0487fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f049009d570 0x7f0490144fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49344/0 (socket says 172.21.15.12:49344) 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f0487fff640 1 -- 172.21.15.12:0/3488879714 learned_addr learned my addr 172.21.15.12:0/3488879714 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f048f7fe640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04900b51b0 0x7f04900af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601189991 0 0) 0x7f049009ce70 con 0x7f049009d570 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.492+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f047c003530 con 0x7f049009d570 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 8229769 0 0) 0x7f047c003530 con 0x7f049009d570 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04900b51b0 msgr2=0x7f04900af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04900b51b0 0x7f04900af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 >> v1:172.21.15.53:6789/0 conn(0x7f049009d9b0 legacy=0x7f049014f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04900afbb0 con 0x7f049009d570 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f0478001680 con 0x7f049009d570 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 >> v1:172.21.15.12:6789/0 conn(0x7f049009d570 legacy=0x7f0490144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:53.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 0x7f047c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04900b51b0 0x7f047c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c004d50 0x7f047c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f049009e770 con 0x7f047c0039f0 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f047c00ba80 con 0x7f047c004d50 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f047c00bc80 con 0x7f04900b51b0 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048f7fe640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 0x7f047c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f0487fff640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04900b51b0 0x7f047c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048ffff640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c004d50 0x7f047c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048f7fe640 1 -- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04900b51b0 msgr2=0x7f047c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048f7fe640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f04900b51b0 0x7f047c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048f7fe640 1 -- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c004d50 msgr2=0x7f047c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048f7fe640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c004d50 0x7f047c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048f7fe640 1 -- 172.21.15.12:0/3488879714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c003530 con 0x7f047c0039f0 2024-04-27T07:04:53.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048ffff640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f047c004d50 0x7f047c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:53.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048f7fe640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 0x7f047c003e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0480009ab0 tx=0x7f048001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94486b9eec03dc9b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800096b0 con 0x7f047c0039f0 2024-04-27T07:04:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04800a6430 con 0x7f047c0039f0 2024-04-27T07:04:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048effd640 1 -- 172.21.15.12:0/3488879714 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800a6820 con 0x7f047c0039f0 2024-04-27T07:04:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f0495c12640 1 -- 172.21.15.12:0/3488879714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 msgr2=0x7f047c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f0495c12640 1 --2- 172.21.15.12:0/3488879714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 0x7f047c003e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0480009ab0 tx=0x7f048001ca50 comp rx=0 tx=0).stop 2024-04-27T07:04:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f048ffff640 1 -- 172.21.15.12:0/3488879714 reap_dead start 2024-04-27T07:04:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f0495c12640 1 -- 172.21.15.12:0/3488879714 shutdown_connections 2024-04-27T07:04:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.496+0000 7f0495c12640 1 -- 172.21.15.12:0/3488879714 >> 172.21.15.12:0/3488879714 conn(0x7f04900b9840 msgr2=0x7f04900b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 -- 172.21.15.12:0/3488879714 shutdown_connections 2024-04-27T07:04:53.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 -- 172.21.15.12:0/3488879714 wait complete. 2024-04-27T07:04:53.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 Processor -- start 2024-04-27T07:04:53.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 -- start start 2024-04-27T07:04:53.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 0x7f049014fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0490148bf0 0x7f0490146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0490072300 0x7f0490070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f049009c670 con 0x7f047c0039f0 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f049009c4f0 con 0x7f0490072300 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f0490151340 con 0x7f0490148bf0 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 0x7f049014fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 0x7f049014fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46958/0 (socket says 172.21.15.12:46958) 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048f7fe640 1 -- 172.21.15.12:0/3952727458 learned_addr learned my addr 172.21.15.12:0/3952727458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0487fff640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0490148bf0 0x7f0490146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048ffff640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0490072300 0x7f0490070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048f7fe640 1 -- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0490148bf0 msgr2=0x7f0490146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048f7fe640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0490148bf0 0x7f0490146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048f7fe640 1 -- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0490072300 msgr2=0x7f0490070510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:53.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048f7fe640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0490072300 0x7f0490070510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:53.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048f7fe640 1 -- 172.21.15.12:0/3952727458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04901484f0 con 0x7f047c0039f0 2024-04-27T07:04:53.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048f7fe640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 0x7f049014fa60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0480009ab0 tx=0x7f047c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:53.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048cff9640 1 -- 172.21.15.12:0/3952727458 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800be070 con 0x7f047c0039f0 2024-04-27T07:04:53.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048cff9640 1 -- 172.21.15.12:0/3952727458 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048000f000 con 0x7f047c0039f0 2024-04-27T07:04:53.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 -- 172.21.15.12:0/3952727458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0490148760 con 0x7f047c0039f0 2024-04-27T07:04:53.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f048cff9640 1 -- 172.21.15.12:0/3952727458 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800b9410 con 0x7f047c0039f0 2024-04-27T07:04:53.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 -- 172.21.15.12:0/3952727458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0490146670 con 0x7f047c0039f0 2024-04-27T07:04:53.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.500+0000 7f0495c12640 1 -- 172.21.15.12:0/3952727458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0490149030 con 0x7f047c0039f0 2024-04-27T07:04:53.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.504+0000 7f048cff9640 1 -- 172.21.15.12:0/3952727458 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f04800c1070 con 0x7f047c0039f0 2024-04-27T07:04:53.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.504+0000 7f048cff9640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f04700683b0 0x7f047006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:53.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.504+0000 7f048cff9640 1 -- 172.21.15.12:0/3952727458 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5357+0+0 (secure 0 0 0) 0x7f04801200e0 con 0x7f047c0039f0 2024-04-27T07:04:53.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.504+0000 7f0487fff640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f04700683b0 0x7f047006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:53.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.504+0000 7f0487fff640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f04700683b0 0x7f047006a870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0478001cd0 tx=0x7f0478009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:53.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.508+0000 7f048cff9640 1 -- 172.21.15.12:0/3952727458 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0490149030 con 0x7f047c0039f0 2024-04-27T07:04:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:53.676+0000 7f0495c12640 1 -- 172.21.15.12:0/3952727458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0490068d00 con 0x7f047c0039f0 2024-04-27T07:04:53.895 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:04:53.898 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:04:54.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.332+0000 7f048cff9640 1 -- 172.21.15.12:0/3952727458 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) ==== 169+0+0 (secure 0 0 0) 0x7f04800ac020 con 0x7f047c0039f0 2024-04-27T07:04:54.342 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2024-04-27T07:04:54.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 -- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f04700683b0 msgr2=0x7f047006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:54.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f04700683b0 0x7f047006a870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0478001cd0 tx=0x7f0478009040 comp rx=0 tx=0).stop 2024-04-27T07:04:54.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 -- 172.21.15.12:0/3952727458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 msgr2=0x7f049014fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:54.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 0x7f049014fa60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0480009ab0 tx=0x7f047c005840 comp rx=0 tx=0).stop 2024-04-27T07:04:54.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 -- 172.21.15.12:0/3952727458 shutdown_connections 2024-04-27T07:04:54.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f04700683b0 0x7f047006a870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:54.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0490072300 0x7f0490070510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:54.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0490148bf0 0x7f0490146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:54.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 --2- 172.21.15.12:0/3952727458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f047c0039f0 0x7f049014fa60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:54.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 -- 172.21.15.12:0/3952727458 >> 172.21.15.12:0/3952727458 conn(0x7f04900b9840 msgr2=0x7f04900b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:54.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 -- 172.21.15.12:0/3952727458 shutdown_connections 2024-04-27T07:04:54.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.336+0000 7f0495c12640 1 -- 172.21.15.12:0/3952727458 wait complete. 2024-04-27T07:04:54.373 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-27T07:04:54.374 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-27T07:04:54.374 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-27T07:04:54.374 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:04:54.374 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-27T07:04:54.374 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-27T07:04:54.374 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-27T07:04:54.374 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:04:54.387 INFO:tasks.ceph.mds.a:Stopped 2024-04-27T07:04:54.387 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-27T07:04:54.389 INFO:tasks.ceph.mds.b:Stopped 2024-04-27T07:04:54.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-27T07:04:54.430 INFO:tasks.ceph.mds.a:Started 2024-04-27T07:04:54.430 DEBUG:teuthology.parallel:result is None 2024-04-27T07:04:54.432 INFO:tasks.ceph.mds.b:Started 2024-04-27T07:04:54.432 DEBUG:teuthology.parallel:result is None 2024-04-27T07:04:54.432 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-27T07:04:54.498 INFO:tasks.ceph.mds.b.smithi053.stdout:starting mds.b at 2024-04-27T07:04:54.499 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2024-04-27T07:04:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf324ed640 1 Processor -- start 2024-04-27T07:04:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf324ed640 1 -- start start 2024-04-27T07:04:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf324ed640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf2c0b6610 0x7faf2c1489d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf324ed640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf2c0b8020 0x7faf2c0b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf324ed640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf2c0b79b0 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf324ed640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7faf2c0b8480 con 0x7faf2c0b8020 2024-04-27T07:04:54.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf324ed640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7faf2c0b69f0 con 0x7faf2c0b6610 2024-04-27T07:04:54.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf2b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf2c0b8020 0x7faf2c0b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:54.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf2b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf2c0b8020 0x7faf2c0b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46976/0 (socket says 172.21.15.12:46976) 2024-04-27T07:04:54.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf30a8a640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7faf2c0b5cf0 0x7faf2c1419d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:51122/0 (socket says 172.21.15.12:51122) 2024-04-27T07:04:54.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf2b7fe640 1 -- 172.21.15.12:0/3466947572 learned_addr learned my addr 172.21.15.12:0/3466947572 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:54.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf2c0b6610 0x7faf2c1489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:54.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf2b7fe640 1 -- 172.21.15.12:0/3466947572 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf2c0b6610 msgr2=0x7faf2c1489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:54.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf2b7fe640 1 --2- 172.21.15.12:0/3466947572 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf2c0b6610 0x7faf2c1489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:54.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf2b7fe640 1 -- 172.21.15.12:0/3466947572 >> v1:172.21.15.53:6789/0 conn(0x7faf2c0b5cf0 legacy=0x7faf2c1419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:54.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.548+0000 7faf2b7fe640 1 -- 172.21.15.12:0/3466947572 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c147ef0 con 0x7faf2c0b8020 2024-04-27T07:04:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf2b7fe640 1 --2- 172.21.15.12:0/3466947572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b8020 0x7faf2c0b57b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faf20009870 tx=0x7faf2001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f1b5be4fe28506 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf2affd640 1 -- 172.21.15.12:0/3466947572 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200b1070 con 0x7faf2c0b8020 2024-04-27T07:04:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf2affd640 1 -- 172.21.15.12:0/3466947572 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf200ac410 con 0x7faf2c0b8020 2024-04-27T07:04:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf2affd640 1 -- 172.21.15.12:0/3466947572 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200ac7e0 con 0x7faf2c0b8020 2024-04-27T07:04:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf324ed640 1 -- 172.21.15.12:0/3466947572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b8020 msgr2=0x7faf2c0b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf324ed640 1 --2- 172.21.15.12:0/3466947572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b8020 0x7faf2c0b57b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faf20009870 tx=0x7faf2001c4a0 comp rx=0 tx=0).stop 2024-04-27T07:04:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf324ed640 1 -- 172.21.15.12:0/3466947572 shutdown_connections 2024-04-27T07:04:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf324ed640 1 --2- 172.21.15.12:0/3466947572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b8020 0x7faf2c0b57b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf324ed640 1 --2- 172.21.15.12:0/3466947572 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf2c0b6610 0x7faf2c1489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf324ed640 1 -- 172.21.15.12:0/3466947572 >> 172.21.15.12:0/3466947572 conn(0x7faf2c0ba140 msgr2=0x7faf2c0ba550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf324ed640 1 -- 172.21.15.12:0/3466947572 shutdown_connections 2024-04-27T07:04:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.552+0000 7faf324ed640 1 -- 172.21.15.12:0/3466947572 wait complete. 2024-04-27T07:04:54.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 Processor -- start 2024-04-27T07:04:54.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 -- start start 2024-04-27T07:04:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b5cf0 0x7faf2c149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf2c0b6610 0x7faf2c14a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf2c0b8020 0x7faf2c149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c155280 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c155100 con 0x7faf2c0b6610 2024-04-27T07:04:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7faf2c155400 con 0x7faf2c0b8020 2024-04-27T07:04:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b5cf0 0x7faf2c149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b5cf0 0x7faf2c149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46988/0 (socket says 172.21.15.12:46988) 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf2b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf2c0b6610 0x7faf2c14a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf2bfff640 1 -- 172.21.15.12:0/229961774 learned_addr learned my addr 172.21.15.12:0/229961774 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf30a8a640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf2c0b8020 0x7faf2c149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf2bfff640 1 -- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf2c0b8020 msgr2=0x7faf2c149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf2bfff640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf2c0b8020 0x7faf2c149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf2bfff640 1 -- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf2c0b6610 msgr2=0x7faf2c14a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf2bfff640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf2c0b6610 0x7faf2c14a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf2bfff640 1 -- 172.21.15.12:0/229961774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c14a770 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf2bfff640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b5cf0 0x7faf2c149170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009db0 tx=0x7faf1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf28ff9640 1 -- 172.21.15.12:0/229961774 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c018020 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf28ff9640 1 -- 172.21.15.12:0/229961774 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf1c014480 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 -- 172.21.15.12:0/229961774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2c14aa50 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf28ff9640 1 -- 172.21.15.12:0/229961774 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c014850 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 -- 172.21.15.12:0/229961774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2c144cc0 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.556+0000 7faf324ed640 1 -- 172.21.15.12:0/229961774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf2c069330 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.560+0000 7faf28ff9640 1 -- 172.21.15.12:0/229961774 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faf1c01b020 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.560+0000 7faf28ff9640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faf180683b0 0x7faf1806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:54.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.560+0000 7faf28ff9640 1 -- 172.21.15.12:0/229961774 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4943+0+0 (secure 0 0 0) 0x7faf1c0876f0 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.560+0000 7faf2b7fe640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faf180683b0 0x7faf1806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:54.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.560+0000 7faf2b7fe640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faf180683b0 0x7faf1806a870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faf200ae000 tx=0x7faf20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:54.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.564+0000 7faf28ff9640 1 -- 172.21.15.12:0/229961774 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faf1c022070 con 0x7faf2c0b5cf0 2024-04-27T07:04:54.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:54.736+0000 7faf324ed640 1 -- 172.21.15.12:0/229961774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7faf2c0b9110 con 0x7faf2c0b5cf0 2024-04-27T07:04:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.368+0000 7faf28ff9640 1 -- 172.21.15.12:0/229961774 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) ==== 97+0+0 (secure 0 0 0) 0x7faf1c010040 con 0x7faf2c0b5cf0 2024-04-27T07:04:55.376 INFO:teuthology.orchestra.run.smithi012.stderr: removed all blocklist entries 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 -- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faf180683b0 msgr2=0x7faf1806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faf180683b0 0x7faf1806a870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faf200ae000 tx=0x7faf20009040 comp rx=0 tx=0).stop 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 -- 172.21.15.12:0/229961774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b5cf0 msgr2=0x7faf2c149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b5cf0 0x7faf2c149170 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009db0 tx=0x7faf1c00c490 comp rx=0 tx=0).stop 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 -- 172.21.15.12:0/229961774 shutdown_connections 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faf180683b0 0x7faf1806a870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7faf2c0b8020 0x7faf2c149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf2c0b6610 0x7faf2c14a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 --2- 172.21.15.12:0/229961774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf2c0b5cf0 0x7faf2c149170 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 -- 172.21.15.12:0/229961774 >> 172.21.15.12:0/229961774 conn(0x7faf2c0ba140 msgr2=0x7faf2c0b8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 -- 172.21.15.12:0/229961774 shutdown_connections 2024-04-27T07:04:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.372+0000 7faf324ed640 1 -- 172.21.15.12:0/229961774 wait complete. 2024-04-27T07:04:55.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-27T07:04:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c92e38640 1 Processor -- start 2024-04-27T07:04:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c92e38640 1 -- start start 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c92e38640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0c8c0b9ae0 0x7f0c8c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c92e38640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0c8c1480a0 0x7f0c8c148500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c92e38640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c8c148a40 con 0x7f0c8c13e5b0 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c92e38640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f0c8c0b9ec0 con 0x7f0c8c0b9ae0 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c92e38640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0c8c09ddb0 con 0x7f0c8c1480a0 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c90bd4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0c8c0b9ae0 0x7f0c8c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c913d5640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0c8c13e5b0 0x7f0c8c13ea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49350/0 (socket says 172.21.15.12:49350) 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c913d5640 1 -- 172.21.15.12:0/1678652487 learned_addr learned my addr 172.21.15.12:0/1678652487 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c8bfff640 1 --2- 172.21.15.12:0/1678652487 >> v2:172.21.15.53:3300/0 conn(0x7f0c8c1480a0 0x7f0c8c148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c8b7fe640 1 -- 172.21.15.12:0/1678652487 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1699658816 0 0) 0x7f0c8c148a40 con 0x7f0c8c13e5b0 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c8b7fe640 1 -- 172.21.15.12:0/1678652487 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c78003530 con 0x7f0c8c13e5b0 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c90bd4640 1 -- 172.21.15.12:0/1678652487 >> v2:172.21.15.53:3300/0 conn(0x7f0c8c1480a0 msgr2=0x7f0c8c148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:55.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c90bd4640 1 --2- 172.21.15.12:0/1678652487 >> v2:172.21.15.53:3300/0 conn(0x7f0c8c1480a0 0x7f0c8c148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c90bd4640 1 -- 172.21.15.12:0/1678652487 >> v1:172.21.15.12:6789/0 conn(0x7f0c8c13e5b0 legacy=0x7f0c8c13ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.520+0000 7f0c90bd4640 1 -- 172.21.15.12:0/1678652487 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c09e090 con 0x7f0c8c0b9ae0 2024-04-27T07:04:55.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c90bd4640 1 --2- 172.21.15.12:0/1678652487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c0b9ae0 0x7f0c8c073c60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c06f030 tx=0x7f0c7c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb4e1d24fdba49 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:55.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c8b7fe640 1 -- 172.21.15.12:0/1678652487 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0b0070 con 0x7f0c8c0b9ae0 2024-04-27T07:04:55.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c8b7fe640 1 -- 172.21.15.12:0/1678652487 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7c0ab600 con 0x7f0c8c0b9ae0 2024-04-27T07:04:55.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c8b7fe640 1 -- 172.21.15.12:0/1678652487 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0ab9d0 con 0x7f0c8c0b9ae0 2024-04-27T07:04:55.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c92e38640 1 -- 172.21.15.12:0/1678652487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c0b9ae0 msgr2=0x7f0c8c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c92e38640 1 --2- 172.21.15.12:0/1678652487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c0b9ae0 0x7f0c8c073c60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c06f030 tx=0x7f0c7c01c480 comp rx=0 tx=0).stop 2024-04-27T07:04:55.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c92e38640 1 -- 172.21.15.12:0/1678652487 shutdown_connections 2024-04-27T07:04:55.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c92e38640 1 --2- 172.21.15.12:0/1678652487 >> v2:172.21.15.53:3300/0 conn(0x7f0c8c1480a0 0x7f0c8c148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c92e38640 1 --2- 172.21.15.12:0/1678652487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c0b9ae0 0x7f0c8c073c60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c92e38640 1 -- 172.21.15.12:0/1678652487 >> 172.21.15.12:0/1678652487 conn(0x7f0c8c067d10 msgr2=0x7f0c8c068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:55.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c92e38640 1 -- 172.21.15.12:0/1678652487 shutdown_connections 2024-04-27T07:04:55.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.524+0000 7f0c92e38640 1 -- 172.21.15.12:0/1678652487 wait complete. 2024-04-27T07:04:55.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 Processor -- start 2024-04-27T07:04:55.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 -- start start 2024-04-27T07:04:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c8c0b9ae0 0x7f0c8c146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0c8c13e5b0 0x7f0c8c146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c1480a0 0x7f0c8c1e3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c0b5110 con 0x7f0c8c1480a0 2024-04-27T07:04:55.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c09db90 con 0x7f0c8c0b9ae0 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8c151cb0 con 0x7f0c8c13e5b0 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c913d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c1480a0 0x7f0c8c1e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c913d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c1480a0 0x7f0c8c1e3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47008/0 (socket says 172.21.15.12:47008) 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c913d5640 1 -- 172.21.15.12:0/3778490820 learned_addr learned my addr 172.21.15.12:0/3778490820 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c90bd4640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c8c0b9ae0 0x7f0c8c146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c8bfff640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0c8c13e5b0 0x7f0c8c146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c913d5640 1 -- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0c8c13e5b0 msgr2=0x7f0c8c146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c913d5640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0c8c13e5b0 0x7f0c8c146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c913d5640 1 -- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c8c0b9ae0 msgr2=0x7f0c8c146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c913d5640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c8c0b9ae0 0x7f0c8c146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c913d5640 1 -- 172.21.15.12:0/3778490820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c0b4cb0 con 0x7f0c8c1480a0 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c90bd4640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c8c0b9ae0 0x7f0c8c146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c913d5640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c1480a0 0x7f0c8c1e3740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400cb30 tx=0x7f0c8400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c897fa640 1 -- 172.21.15.12:0/3778490820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c84017070 con 0x7f0c8c1480a0 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c897fa640 1 -- 172.21.15.12:0/3778490820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c8400aca0 con 0x7f0c8c1480a0 2024-04-27T07:04:55.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 -- 172.21.15.12:0/3778490820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8c13f590 con 0x7f0c8c1480a0 2024-04-27T07:04:55.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c897fa640 1 -- 172.21.15.12:0/3778490820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c84012600 con 0x7f0c8c1480a0 2024-04-27T07:04:55.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 -- 172.21.15.12:0/3778490820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8c147b90 con 0x7f0c8c1480a0 2024-04-27T07:04:55.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c92e38640 1 -- 172.21.15.12:0/3778490820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8c140790 con 0x7f0c8c1480a0 2024-04-27T07:04:55.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.528+0000 7f0c897fa640 1 -- 172.21.15.12:0/3778490820 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0c8401a070 con 0x7f0c8c1480a0 2024-04-27T07:04:55.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.532+0000 7f0c897fa640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0c6c068380 0x7f0c6c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.532+0000 7f0c897fa640 1 -- 172.21.15.12:0/3778490820 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f0c84086310 con 0x7f0c8c1480a0 2024-04-27T07:04:55.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.532+0000 7f0c90bd4640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0c6c068380 0x7f0c6c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.532+0000 7f0c90bd4640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0c6c068380 0x7f0c6c06a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0ad000 tx=0x7f0c7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.536+0000 7f0c897fa640 1 -- 172.21.15.12:0/3778490820 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0c8c140790 con 0x7f0c8c1480a0 2024-04-27T07:04:55.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 -- 172.21.15.12:0/3778490820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0c8c068c20 con 0x7f0c8c1480a0 2024-04-27T07:04:55.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c897fa640 1 -- 172.21.15.12:0/3778490820 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+5207 (secure 0 0 0) 0x7f0c84056660 con 0x7f0c8c1480a0 2024-04-27T07:04:55.761 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:55.761 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:04:55.761 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_dump": [ 2024-04-27T07:04:55.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.761 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "mds.a", 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQA+oyxmeosVBxAA64Na7IWgq/9JX8IryFKwqQ==", 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow *", 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow *", 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "mds.b", 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQA+oyxm9+xNDRAAhFXmvJv+YKWpQa4MG10lPA==", 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2024-04-27T07:04:55.762 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow *", 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow *", 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.0", 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBBoyxm9cBBEhAA+7TsG1DI1x9B58lFniRTCA==", 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.1", 2024-04-27T07:04:55.763 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBEoyxmUkfXFxAAeOVSfqw19vxB9lh6A+B2mw==", 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.2", 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBHoyxm05j6HRAASwIItsLH5QWhSJJua+scaQ==", 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.3", 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBKoyxmIUTjJRAAXn6H3GX7kIiyF6rxixzGxg==", 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.4", 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBOoyxm+BeLKhAAgVHFVXecbZN0h4L6JR6i2A==", 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.5", 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBRoyxmCoyNMBAATx0/vNa76EqzZVzqe7aEVg==", 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.6", 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBUoyxmMtpIOBAACHVu23d6W1P51eipJYpUzg==", 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2024-04-27T07:04:55.766 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "osd.7", 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBYoyxm5EfTAxAA004KGJLbiMGRzrS9Q9PofQ==", 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow profile osd", 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile osd", 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.0", 2024-04-27T07:04:55.767 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQA+oyxmgrAhExAAceKsj8G+Q/XXODrJbKPZ0Q==", 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow", 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow r", 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow rw", 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow rwx" 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.admin", 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQA9oyxmZEtoKRAAi/4qaAB8T6yCI3GYTckrgw==", 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow *", 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": "allow *", 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow *", 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.768 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-mds", 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBdoyxmvmymKhAAi7AtmTM3KfHsUw5z0oDfpA==", 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-mds" 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-mgr", 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBdoyxm/4emKhAAa3BEuu5N83i/RJIkzVYTkA==", 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-osd", 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBdoyxm0KCmKhAAGrKxKLauUAf+6EKOVebbsA==", 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-osd" 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-rbd", 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBdoyxmQbmmKhAAid4vXWLcYz4rCYPKoXflgA==", 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.770 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBdoyxml9GmKhAAcxN9wwI5aBQyQ6JFzWOVmQ==", 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "client.bootstrap-rgw", 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQBdoyxmpOqmKhAAGbcYrUVBWI67N+7uqpWgHQ==", 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "mgr.x", 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQA+oyxmryeZAxAAbc4+8Txh5jr/zMCfbmKqtw==", 2024-04-27T07:04:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow *", 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile mgr", 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: "entity": "mgr.y", 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "AQA+oyxm+NugABAAzPlk/x6yFXaTG5J0uLdcOA==", 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": { 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": "allow *", 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: "mon": "allow profile mgr", 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "allow *" 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-27T07:04:55.772 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 -- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0c6c068380 msgr2=0x7f0c6c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0c6c068380 0x7f0c6c06a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0ad000 tx=0x7f0c7c008040 comp rx=0 tx=0).stop 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 -- 172.21.15.12:0/3778490820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c1480a0 msgr2=0x7f0c8c1e3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c1480a0 0x7f0c8c1e3740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400cb30 tx=0x7f0c8400a060 comp rx=0 tx=0).stop 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 -- 172.21.15.12:0/3778490820 shutdown_connections 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0c6c068380 0x7f0c6c06a840 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c8c1480a0 0x7f0c8c1e3740 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0c8c13e5b0 0x7f0c8c146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 --2- 172.21.15.12:0/3778490820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c8c0b9ae0 0x7f0c8c146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 -- 172.21.15.12:0/3778490820 >> 172.21.15.12:0/3778490820 conn(0x7f0c8c067d10 msgr2=0x7f0c8c06db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 -- 172.21.15.12:0/3778490820 shutdown_connections 2024-04-27T07:04:55.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.756+0000 7f0c92e38640 1 -- 172.21.15.12:0/3778490820 wait complete. 2024-04-27T07:04:55.780 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-27T07:04:55.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff8a504f640 1 Processor -- start 2024-04-27T07:04:55.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff8a504f640 1 -- start start 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff8a504f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff8a006bde0 0x7ff8a00af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff8a504f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8a00afe10 con 0x7ff8a014a800 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff8a504f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8a006f450 con 0x7ff8a00b12f0 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff8a504f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a006c1c0 con 0x7ff8a006bde0 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff89ed9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff8a006bde0 0x7ff8a00af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff89f59e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff8a014a800 0x7ff8a0152600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49364/0 (socket says 172.21.15.12:49364) 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff89f59e640 1 -- 172.21.15.12:0/3890605605 learned_addr learned my addr 172.21.15.12:0/3890605605 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff89ed9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff8a006bde0 0x7ff8a00af510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47012/0 (socket says 172.21.15.12:47012) 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff89ed9d640 1 -- 172.21.15.12:0/3890605605 >> v1:172.21.15.53:6789/0 conn(0x7ff8a00b12f0 legacy=0x7ff8a0148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff89ed9d640 1 -- 172.21.15.12:0/3890605605 >> v1:172.21.15.12:6789/0 conn(0x7ff8a014a800 legacy=0x7ff8a0152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.896+0000 7ff89ed9d640 1 -- 172.21.15.12:0/3890605605 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a00afb30 con 0x7ff8a006bde0 2024-04-27T07:04:55.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff89ed9d640 1 --2- 172.21.15.12:0/3890605605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a006bde0 0x7ff8a00af510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff890009d90 tx=0x7ff89001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c1fb364f9fe68f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:55.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff89dd9b640 1 -- 172.21.15.12:0/3890605605 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8900b2070 con 0x7ff8a006bde0 2024-04-27T07:04:55.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff89dd9b640 1 -- 172.21.15.12:0/3890605605 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8900a5760 con 0x7ff8a006bde0 2024-04-27T07:04:55.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff89dd9b640 1 -- 172.21.15.12:0/3890605605 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8900a5b30 con 0x7ff8a006bde0 2024-04-27T07:04:55.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff8a504f640 1 -- 172.21.15.12:0/3890605605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a006bde0 msgr2=0x7ff8a00af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff8a504f640 1 --2- 172.21.15.12:0/3890605605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a006bde0 0x7ff8a00af510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff890009d90 tx=0x7ff89001c8a0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff8a504f640 1 -- 172.21.15.12:0/3890605605 shutdown_connections 2024-04-27T07:04:55.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff8a504f640 1 --2- 172.21.15.12:0/3890605605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a006bde0 0x7ff8a00af510 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff8a504f640 1 -- 172.21.15.12:0/3890605605 >> 172.21.15.12:0/3890605605 conn(0x7ff8a00b6ff0 msgr2=0x7ff8a00b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:55.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff8a504f640 1 -- 172.21.15.12:0/3890605605 shutdown_connections 2024-04-27T07:04:55.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff8a504f640 1 -- 172.21.15.12:0/3890605605 wait complete. 2024-04-27T07:04:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.900+0000 7ff8a504f640 1 Processor -- start 2024-04-27T07:04:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8a504f640 1 -- start start 2024-04-27T07:04:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8a504f640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8a006bde0 0x7ff8a00b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8a504f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a00b12f0 0x7ff8a01497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8a504f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a014a800 0x7ff8a00b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8a504f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01566c0 con 0x7ff8a00b12f0 2024-04-27T07:04:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8a504f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a0156540 con 0x7ff8a014a800 2024-04-27T07:04:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8a504f640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a0156840 con 0x7ff8a006bde0 2024-04-27T07:04:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a00b12f0 0x7ff8a01497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a00b12f0 0x7ff8a01497d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47018/0 (socket says 172.21.15.12:47018) 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89e59c640 1 -- 172.21.15.12:0/3246727854 learned_addr learned my addr 172.21.15.12:0/3246727854 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89ed9d640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8a006bde0 0x7ff8a00b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89f59e640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a014a800 0x7ff8a00b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89e59c640 1 -- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8a006bde0 msgr2=0x7ff8a00b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89e59c640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8a006bde0 0x7ff8a00b60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89e59c640 1 -- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a014a800 msgr2=0x7ff8a00b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89e59c640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a014a800 0x7ff8a00b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89e59c640 1 -- 172.21.15.12:0/3246727854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a0071920 con 0x7ff8a00b12f0 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89e59c640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a00b12f0 0x7ff8a01497d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff89400c450 tx=0x7ff89400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8877fe640 1 -- 172.21.15.12:0/3246727854 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894012830 con 0x7ff8a00b12f0 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8877fe640 1 -- 172.21.15.12:0/3246727854 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff894017070 con 0x7ff8a00b12f0 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8877fe640 1 -- 172.21.15.12:0/3246727854 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894012c30 con 0x7ff8a00b12f0 2024-04-27T07:04:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8a504f640 1 -- 172.21.15.12:0/3246727854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a0071c00 con 0x7ff8a00b12f0 2024-04-27T07:04:55.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8a504f640 1 -- 172.21.15.12:0/3246727854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a0072090 con 0x7ff8a00b12f0 2024-04-27T07:04:55.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8a504f640 1 -- 172.21.15.12:0/3246727854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a0069330 con 0x7ff8a00b12f0 2024-04-27T07:04:55.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8877fe640 1 -- 172.21.15.12:0/3246727854 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff89401c460 con 0x7ff8a00b12f0 2024-04-27T07:04:55.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8877fe640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff88c0683b0 0x7ff88c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:55.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff8877fe640 1 -- 172.21.15.12:0/3246727854 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7ff8940863c0 con 0x7ff8a00b12f0 2024-04-27T07:04:55.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.904+0000 7ff89ed9d640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff88c0683b0 0x7ff88c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:55.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.908+0000 7ff89ed9d640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff88c0683b0 0x7ff88c06a870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff8900af000 tx=0x7ff8900a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:55.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:55.908+0000 7ff8877fe640 1 -- 172.21.15.12:0/3246727854 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff894056d60 con 0x7ff8a00b12f0 2024-04-27T07:04:56.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.132+0000 7ff8a504f640 1 -- 172.21.15.12:0/3246727854 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7ff8a00b4990 con 0x7ff8a00b12f0 2024-04-27T07:04:56.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.136+0000 7ff8877fe640 1 -- 172.21.15.12:0/3246727854 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7ff894056660 con 0x7ff8a00b12f0 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 -- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff88c0683b0 msgr2=0x7ff88c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff88c0683b0 0x7ff88c06a870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff8900af000 tx=0x7ff8900a7070 comp rx=0 tx=0).stop 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 -- 172.21.15.12:0/3246727854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a00b12f0 msgr2=0x7ff8a01497d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a00b12f0 0x7ff8a01497d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff89400c450 tx=0x7ff89400c910 comp rx=0 tx=0).stop 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 -- 172.21.15.12:0/3246727854 shutdown_connections 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff88c0683b0 0x7ff88c06a870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8a014a800 0x7ff8a00b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8a00b12f0 0x7ff8a01497d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 --2- 172.21.15.12:0/3246727854 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8a006bde0 0x7ff8a00b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 -- 172.21.15.12:0/3246727854 >> 172.21.15.12:0/3246727854 conn(0x7ff8a00b6ff0 msgr2=0x7ff8a009bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 -- 172.21.15.12:0/3246727854 shutdown_connections 2024-04-27T07:04:56.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.140+0000 7ff8a504f640 1 -- 172.21.15.12:0/3246727854 wait complete. 2024-04-27T07:04:56.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-27T07:04:56.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed79e55640 1 Processor -- start 2024-04-27T07:04:56.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed79e55640 1 -- start start 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed79e55640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed7406e370 0x7fed7413faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed79e55640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed7406d300 0x7fed74150420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed79e55640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed740b4d70 con 0x7fed740b3650 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed79e55640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fed740740d0 con 0x7fed7406e370 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed79e55640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fed740b2bf0 con 0x7fed7406d300 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed72ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fed740b3650 0x7fed740b5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49378/0 (socket says 172.21.15.12:49378) 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed72ffd640 1 -- 172.21.15.12:0/3962280193 learned_addr learned my addr 172.21.15.12:0/3962280193 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed737fe640 1 --2- 172.21.15.12:0/3962280193 >> v2:172.21.15.53:3300/0 conn(0x7fed7406e370 0x7fed7413faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed73fff640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed7406d300 0x7fed74150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335706601 0 0) 0x7fed740b4d70 con 0x7fed740b3650 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed5c003530 con 0x7fed740b3650 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 253705212 0 0) 0x7fed5c003530 con 0x7fed740b3650 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed7406d300 msgr2=0x7fed74150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:56.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed7406d300 0x7fed74150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 >> v2:172.21.15.53:3300/0 conn(0x7fed7406e370 msgr2=0x7fed7413faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 --2- 172.21.15.12:0/3962280193 >> v2:172.21.15.53:3300/0 conn(0x7fed7406e370 0x7fed7413faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed74073e60 con 0x7fed740b3650 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fed68001680 con 0x7fed740b3650 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 >> v1:172.21.15.12:6789/0 conn(0x7fed740b3650 legacy=0x7fed740b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed7406d300 0x7fed5c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5c003f40 0x7fed5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed5c004d50 0x7fed5c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fed740740d0 con 0x7fed5c003f40 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed740b2bf0 con 0x7fed5c004d50 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fed5c00ba20 con 0x7fed7406d300 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed72ffd640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5c003f40 0x7fed5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed737fe640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed7406d300 0x7fed5c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed72ffd640 1 -- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed7406d300 msgr2=0x7fed5c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed73fff640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed5c004d50 0x7fed5c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed72ffd640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed7406d300 0x7fed5c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed72ffd640 1 -- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed5c004d50 msgr2=0x7fed5c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed72ffd640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed5c004d50 0x7fed5c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed72ffd640 1 -- 172.21.15.12:0/3962280193 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c003530 con 0x7fed5c003f40 2024-04-27T07:04:56.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.284+0000 7fed737fe640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed7406d300 0x7fed5c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed72ffd640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5c003f40 0x7fed5c004810 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed740b3a70 tx=0x7fed6801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc71fe994e459e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68007d80 con 0x7fed5c003f40 2024-04-27T07:04:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed680a9020 con 0x7fed5c003f40 2024-04-27T07:04:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed727fc640 1 -- 172.21.15.12:0/3962280193 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed680058e0 con 0x7fed5c003f40 2024-04-27T07:04:56.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 -- 172.21.15.12:0/3962280193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5c003f40 msgr2=0x7fed5c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 --2- 172.21.15.12:0/3962280193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed5c003f40 0x7fed5c004810 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed740b3a70 tx=0x7fed6801cc60 comp rx=0 tx=0).stop 2024-04-27T07:04:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed73fff640 1 -- 172.21.15.12:0/3962280193 reap_dead start 2024-04-27T07:04:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 -- 172.21.15.12:0/3962280193 shutdown_connections 2024-04-27T07:04:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 -- 172.21.15.12:0/3962280193 >> 172.21.15.12:0/3962280193 conn(0x7fed740b93b0 msgr2=0x7fed740b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 -- 172.21.15.12:0/3962280193 shutdown_connections 2024-04-27T07:04:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 -- 172.21.15.12:0/3962280193 wait complete. 2024-04-27T07:04:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 Processor -- start 2024-04-27T07:04:56.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 -- start start 2024-04-27T07:04:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed5c003f40 0x7fed740711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed740705d0 0x7fed74072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed741dc470 0x7fed7413a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fed7414efb0 con 0x7fed741dc470 2024-04-27T07:04:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed7414ee30 con 0x7fed740705d0 2024-04-27T07:04:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed79e55640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fed7414f130 con 0x7fed5c003f40 2024-04-27T07:04:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed73fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed741dc470 0x7fed7413a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed73fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed741dc470 0x7fed7413a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47044/0 (socket says 172.21.15.12:47044) 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.288+0000 7fed73fff640 1 -- 172.21.15.12:0/3322382465 learned_addr learned my addr 172.21.15.12:0/3322382465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed72ffd640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed740705d0 0x7fed74072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed737fe640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed5c003f40 0x7fed740711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed73fff640 1 -- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed5c003f40 msgr2=0x7fed740711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed73fff640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed5c003f40 0x7fed740711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed73fff640 1 -- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed740705d0 msgr2=0x7fed74072cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed73fff640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed740705d0 0x7fed74072cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed73fff640 1 -- 172.21.15.12:0/3322382465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed74146600 con 0x7fed741dc470 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed73fff640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed741dc470 0x7fed7413a120 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fed7406d730 tx=0x7fed6c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed5bfff640 1 -- 172.21.15.12:0/3322382465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c01c070 con 0x7fed741dc470 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed5bfff640 1 -- 172.21.15.12:0/3322382465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6c017400 con 0x7fed741dc470 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed79e55640 1 -- 172.21.15.12:0/3322382465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed741468c0 con 0x7fed741dc470 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed5bfff640 1 -- 172.21.15.12:0/3322382465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0177f0 con 0x7fed741dc470 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed79e55640 1 -- 172.21.15.12:0/3322382465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed74072550 con 0x7fed741dc470 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed79e55640 1 -- 172.21.15.12:0/3322382465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed74146430 con 0x7fed741dc470 2024-04-27T07:04:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed5bfff640 1 -- 172.21.15.12:0/3322382465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fed6c01f070 con 0x7fed741dc470 2024-04-27T07:04:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed5bfff640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fed60068380 0x7fed6006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed5bfff640 1 -- 172.21.15.12:0/3322382465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fed6c0868d0 con 0x7fed741dc470 2024-04-27T07:04:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed737fe640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fed60068380 0x7fed6006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.292+0000 7fed737fe640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fed60068380 0x7fed6006a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed64009b40 tx=0x7fed64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.296+0000 7fed5bfff640 1 -- 172.21.15.12:0/3322382465 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fed6c012070 con 0x7fed741dc470 2024-04-27T07:04:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.516+0000 7fed79e55640 1 -- 172.21.15.12:0/3322382465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fed7406dd90 con 0x7fed741dc470 2024-04-27T07:04:56.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.520+0000 7fed5bfff640 1 -- 172.21.15.12:0/3322382465 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fed6c057270 con 0x7fed741dc470 2024-04-27T07:04:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 -- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fed60068380 msgr2=0x7fed6006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fed60068380 0x7fed6006a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fed64009b40 tx=0x7fed64008040 comp rx=0 tx=0).stop 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 -- 172.21.15.12:0/3322382465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed741dc470 msgr2=0x7fed7413a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed741dc470 0x7fed7413a120 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fed7406d730 tx=0x7fed6c00f4a0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 -- 172.21.15.12:0/3322382465 shutdown_connections 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fed60068380 0x7fed6006a840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed741dc470 0x7fed7413a120 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed740705d0 0x7fed74072cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 --2- 172.21.15.12:0/3322382465 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed5c003f40 0x7fed740711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 -- 172.21.15.12:0/3322382465 >> 172.21.15.12:0/3322382465 conn(0x7fed740b93b0 msgr2=0x7fed740ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 -- 172.21.15.12:0/3322382465 shutdown_connections 2024-04-27T07:04:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.524+0000 7fed79e55640 1 -- 172.21.15.12:0/3322382465 wait complete. 2024-04-27T07:04:56.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-27T07:04:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa58922d640 1 Processor -- start 2024-04-27T07:04:56.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa58922d640 1 -- start start 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa58922d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa5840b6480 0x7fa5840b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa58922d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fa584142010 0x7fa5840b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa58922d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5840b6860 con 0x7fa5841489c0 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa58922d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fa5840b84e0 con 0x7fa5840b6480 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa58922d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fa584147ee0 con 0x7fa584142010 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa582d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa5840b6480 0x7fa5840b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa582d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa5840b6480 0x7fa5840b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47050/0 (socket says 172.21.15.12:47050) 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa582d9d640 1 -- 172.21.15.12:0/3459637141 learned_addr learned my addr 172.21.15.12:0/3459637141 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa58359e640 1 --2- 172.21.15.12:0/3459637141 >> v2:172.21.15.53:3300/0 conn(0x7fa584142010 0x7fa5840b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa582d9d640 1 -- 172.21.15.12:0/3459637141 >> v2:172.21.15.53:3300/0 conn(0x7fa584142010 msgr2=0x7fa5840b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa582d9d640 1 --2- 172.21.15.12:0/3459637141 >> v2:172.21.15.53:3300/0 conn(0x7fa584142010 0x7fa5840b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa582d9d640 1 -- 172.21.15.12:0/3459637141 >> v1:172.21.15.53:6789/0 conn(0x7fa5841489c0 legacy=0x7fa584141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:56.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.660+0000 7fa582d9d640 1 -- 172.21.15.12:0/3459637141 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5841481c0 con 0x7fa5840b6480 2024-04-27T07:04:56.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa582d9d640 1 --2- 172.21.15.12:0/3459637141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840b6480 0x7fa5840b5ae0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa5840b55b0 tx=0x7fa57401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeae24501af07e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:56.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa581d9b640 1 -- 172.21.15.12:0/3459637141 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740b0070 con 0x7fa5840b6480 2024-04-27T07:04:56.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa581d9b640 1 -- 172.21.15.12:0/3459637141 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5740ab600 con 0x7fa5840b6480 2024-04-27T07:04:56.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa581d9b640 1 -- 172.21.15.12:0/3459637141 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740ab9d0 con 0x7fa5840b6480 2024-04-27T07:04:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 -- 172.21.15.12:0/3459637141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840b6480 msgr2=0x7fa5840b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 --2- 172.21.15.12:0/3459637141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840b6480 0x7fa5840b5ae0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa5840b55b0 tx=0x7fa57401c450 comp rx=0 tx=0).stop 2024-04-27T07:04:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 -- 172.21.15.12:0/3459637141 shutdown_connections 2024-04-27T07:04:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 --2- 172.21.15.12:0/3459637141 >> v2:172.21.15.53:3300/0 conn(0x7fa584142010 0x7fa5840b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 --2- 172.21.15.12:0/3459637141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5840b6480 0x7fa5840b5ae0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 -- 172.21.15.12:0/3459637141 >> 172.21.15.12:0/3459637141 conn(0x7fa5840bd620 msgr2=0x7fa5840bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 -- 172.21.15.12:0/3459637141 shutdown_connections 2024-04-27T07:04:56.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 -- 172.21.15.12:0/3459637141 wait complete. 2024-04-27T07:04:56.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 Processor -- start 2024-04-27T07:04:56.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 -- start start 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5840b6480 0x7fa58413d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa584142010 0x7fa5840bcab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5841489c0 0x7fa5841389a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa584154e70 con 0x7fa584142010 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa584142420 con 0x7fa5840b6480 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58922d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fa584154ff0 con 0x7fa5841489c0 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa584142010 0x7fa5840bcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa584142010 0x7fa5840bcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47054/0 (socket says 172.21.15.12:47054) 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58259c640 1 -- 172.21.15.12:0/722397677 learned_addr learned my addr 172.21.15.12:0/722397677 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa582d9d640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5840b6480 0x7fa58413d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58359e640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5841489c0 0x7fa5841389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58259c640 1 -- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5841489c0 msgr2=0x7fa5841389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58259c640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5841489c0 0x7fa5841389a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58259c640 1 -- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5840b6480 msgr2=0x7fa58413d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58259c640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5840b6480 0x7fa58413d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa58259c640 1 -- 172.21.15.12:0/722397677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5841521a0 con 0x7fa584142010 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.664+0000 7fa582d9d640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5840b6480 0x7fa58413d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa58259c640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa584142010 0x7fa5840bcab0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa57800c480 tx=0x7fa57800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa56b7fe640 1 -- 172.21.15.12:0/722397677 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578012830 con 0x7fa584142010 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa56b7fe640 1 -- 172.21.15.12:0/722397677 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa578017070 con 0x7fa584142010 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa58922d640 1 -- 172.21.15.12:0/722397677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa584139d80 con 0x7fa584142010 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa56b7fe640 1 -- 172.21.15.12:0/722397677 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578012c30 con 0x7fa584142010 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa58922d640 1 -- 172.21.15.12:0/722397677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58413ce60 con 0x7fa584142010 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa58922d640 1 -- 172.21.15.12:0/722397677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa58413d060 con 0x7fa584142010 2024-04-27T07:04:56.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa56b7fe640 1 -- 172.21.15.12:0/722397677 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa57801c460 con 0x7fa584142010 2024-04-27T07:04:56.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa56b7fe640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa5700683b0 0x7fa57006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:56.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa56b7fe640 1 -- 172.21.15.12:0/722397677 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fa578086450 con 0x7fa584142010 2024-04-27T07:04:56.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa582d9d640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa5700683b0 0x7fa57006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:56.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.668+0000 7fa582d9d640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa5700683b0 0x7fa57006a870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5740ad000 tx=0x7fa574008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:56.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.672+0000 7fa56b7fe640 1 -- 172.21.15.12:0/722397677 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa58413d060 con 0x7fa584142010 2024-04-27T07:04:56.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.884+0000 7fa58922d640 1 -- 172.21.15.12:0/722397677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fa584068c20 con 0x7fa584142010 2024-04-27T07:04:56.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.888+0000 7fa56b7fe640 1 -- 172.21.15.12:0/722397677 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fa578056d50 con 0x7fa584142010 2024-04-27T07:04:56.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 -- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa5700683b0 msgr2=0x7fa57006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa5700683b0 0x7fa57006a870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5740ad000 tx=0x7fa574008040 comp rx=0 tx=0).stop 2024-04-27T07:04:56.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 -- 172.21.15.12:0/722397677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa584142010 msgr2=0x7fa5840bcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:56.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa584142010 0x7fa5840bcab0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa57800c480 tx=0x7fa57800c940 comp rx=0 tx=0).stop 2024-04-27T07:04:56.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 -- 172.21.15.12:0/722397677 shutdown_connections 2024-04-27T07:04:56.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa5700683b0 0x7fa57006a870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5841489c0 0x7fa5841389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa584142010 0x7fa5840bcab0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 --2- 172.21.15.12:0/722397677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5840b6480 0x7fa58413d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:56.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 -- 172.21.15.12:0/722397677 >> 172.21.15.12:0/722397677 conn(0x7fa5840bd620 msgr2=0x7fa58413ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:56.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 -- 172.21.15.12:0/722397677 shutdown_connections 2024-04-27T07:04:56.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:56.892+0000 7fa58922d640 1 -- 172.21.15.12:0/722397677 wait complete. 2024-04-27T07:04:56.912 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-27T07:04:57.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4addab640 1 Processor -- start 2024-04-27T07:04:57.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4addab640 1 -- start start 2024-04-27T07:04:57.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4addab640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc4a80b3db0 0x7fc4a809da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4addab640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a8144cb0 con 0x7fc4a80b4ae0 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4addab640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a8144f90 con 0x7fc4a806e410 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4addab640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a8144a80 con 0x7fc4a80b3db0 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a77fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc4a80b4ae0 0x7fc4a81454b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49384/0 (socket says 172.21.15.12:49384) 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a77fe640 1 -- 172.21.15.12:0/333436253 learned_addr learned my addr 172.21.15.12:0/333436253 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a6ffd640 1 --2- 172.21.15.12:0/333436253 >> v2:172.21.15.53:3300/0 conn(0x7fc4a80b3db0 0x7fc4a809da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712181162 0 0) 0x7fc4a8144cb0 con 0x7fc4a80b4ae0 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc490003530 con 0x7fc4a80b4ae0 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4221990971 0 0) 0x7fc490003530 con 0x7fc4a80b4ae0 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 >> v2:172.21.15.53:3300/0 conn(0x7fc4a80b3db0 msgr2=0x7fc4a809da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 --2- 172.21.15.12:0/333436253 >> v2:172.21.15.53:3300/0 conn(0x7fc4a80b3db0 0x7fc4a809da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 >> v1:172.21.15.53:6789/0 conn(0x7fc4a806e410 legacy=0x7fc4a814f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a8073de0 con 0x7fc4a80b4ae0 2024-04-27T07:04:57.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fc4a0001a00 con 0x7fc4a80b4ae0 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 >> v1:172.21.15.12:6789/0 conn(0x7fc4a80b4ae0 legacy=0x7fc4a81454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc4900039c0 0x7fc490003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc490004810 0x7fc490004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4900051d0 0x7fc49000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc49000c040 con 0x7fc4900051d0 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc49000c1f0 con 0x7fc490004810 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.028+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc49000e600 con 0x7fc4900039c0 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a7fff640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4900051d0 0x7fc49000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a77fe640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc4900039c0 0x7fc490003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a6ffd640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc490004810 0x7fc490004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a7fff640 1 -- 172.21.15.12:0/333436253 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc4900039c0 msgr2=0x7fc490003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a7fff640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc4900039c0 0x7fc490003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a7fff640 1 -- 172.21.15.12:0/333436253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc490004810 msgr2=0x7fc490004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a7fff640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc490004810 0x7fc490004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a7fff640 1 -- 172.21.15.12:0/333436253 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc490003530 con 0x7fc4900051d0 2024-04-27T07:04:57.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a7fff640 1 -- 172.21.15.12:0/333436253 reap_dead start 2024-04-27T07:04:57.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a7fff640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4900051d0 0x7fc49000b9d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80b7280 tx=0x7fc49801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b3ded6bb06d880 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:57.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980013a0 con 0x7fc4900051d0 2024-04-27T07:04:57.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4980052d0 con 0x7fc4900051d0 2024-04-27T07:04:57.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4a67fc640 1 -- 172.21.15.12:0/333436253 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498003840 con 0x7fc4900051d0 2024-04-27T07:04:57.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4addab640 1 -- 172.21.15.12:0/333436253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4900051d0 msgr2=0x7fc49000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4addab640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4900051d0 0x7fc49000b9d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80b7280 tx=0x7fc49801c3c0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4addab640 1 -- 172.21.15.12:0/333436253 shutdown_connections 2024-04-27T07:04:57.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4addab640 1 --2- 172.21.15.12:0/333436253 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4900051d0 0x7fc49000b9d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4addab640 1 -- 172.21.15.12:0/333436253 >> 172.21.15.12:0/333436253 conn(0x7fc4a80b93b0 msgr2=0x7fc4a80b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:57.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4addab640 1 -- 172.21.15.12:0/333436253 shutdown_connections 2024-04-27T07:04:57.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.032+0000 7fc4addab640 1 -- 172.21.15.12:0/333436253 wait complete. 2024-04-27T07:04:57.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 Processor -- start 2024-04-27T07:04:57.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 -- start start 2024-04-27T07:04:57.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4900051d0 0x7fc4a81cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc4a81ccd40 0x7fc4a81dc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4a81cd1c0 0x7fc4a81dce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a8145840 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81456c0 con 0x7fc4900051d0 2024-04-27T07:04:57.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a81459c0 con 0x7fc4a81ccd40 2024-04-27T07:04:57.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4a81cd1c0 0x7fc4a81dce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4a81cd1c0 0x7fc4a81dce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47066/0 (socket says 172.21.15.12:47066) 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a7fff640 1 -- 172.21.15.12:0/4012270350 learned_addr learned my addr 172.21.15.12:0/4012270350 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a6ffd640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc4a81ccd40 0x7fc4a81dc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a77fe640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4900051d0 0x7fc4a81cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a7fff640 1 -- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc4a81ccd40 msgr2=0x7fc4a81dc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a7fff640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc4a81ccd40 0x7fc4a81dc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a7fff640 1 -- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4900051d0 msgr2=0x7fc4a81cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a7fff640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4900051d0 0x7fc4a81cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a7fff640 1 -- 172.21.15.12:0/4012270350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a81cb1b0 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4a7fff640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4a81cd1c0 0x7fc4a81dce30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc49801c1f0 tx=0x7fc490005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc48ffff640 1 -- 172.21.15.12:0/4012270350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980013a0 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc48ffff640 1 -- 172.21.15.12:0/4012270350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc498005480 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc48ffff640 1 -- 172.21.15.12:0/4012270350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498007a40 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 -- 172.21.15.12:0/4012270350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a81cd6a0 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 -- 172.21.15.12:0/4012270350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a80b8690 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc4addab640 1 -- 172.21.15.12:0/4012270350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a81cafe0 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc48ffff640 1 -- 172.21.15.12:0/4012270350 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc498007240 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.036+0000 7fc48ffff640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc494068380 0x7fc49406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.040+0000 7fc48ffff640 1 -- 172.21.15.12:0/4012270350 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fc49811fda0 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.040+0000 7fc4a77fe640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc494068380 0x7fc49406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.040+0000 7fc4a77fe640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc494068380 0x7fc49406a840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0000f90 tx=0x7fc4a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:57.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.044+0000 7fc48ffff640 1 -- 172.21.15.12:0/4012270350 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4980160d0 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.260+0000 7fc4addab640 1 -- 172.21.15.12:0/4012270350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fc4a8068c20 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.264+0000 7fc48ffff640 1 -- 172.21.15.12:0/4012270350 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fc4980f0740 con 0x7fc4a81cd1c0 2024-04-27T07:04:57.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 -- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc494068380 msgr2=0x7fc49406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc494068380 0x7fc49406a840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0000f90 tx=0x7fc4a0009040 comp rx=0 tx=0).stop 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 -- 172.21.15.12:0/4012270350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4a81cd1c0 msgr2=0x7fc4a81dce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4a81cd1c0 0x7fc4a81dce30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc49801c1f0 tx=0x7fc490005840 comp rx=0 tx=0).stop 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 -- 172.21.15.12:0/4012270350 shutdown_connections 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc494068380 0x7fc49406a840 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4a81cd1c0 0x7fc4a81dce30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc4a81ccd40 0x7fc4a81dc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 --2- 172.21.15.12:0/4012270350 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4900051d0 0x7fc4a81cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 -- 172.21.15.12:0/4012270350 >> 172.21.15.12:0/4012270350 conn(0x7fc4a80b93b0 msgr2=0x7fc4a80b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 -- 172.21.15.12:0/4012270350 shutdown_connections 2024-04-27T07:04:57.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.268+0000 7fc4addab640 1 -- 172.21.15.12:0/4012270350 wait complete. 2024-04-27T07:04:57.290 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-27T07:04:57.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c17bb2640 1 Processor -- start 2024-04-27T07:04:57.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c17bb2640 1 -- start start 2024-04-27T07:04:57.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c17bb2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8c10141670 0x7f8c10141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c17bb2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c1009d440 0x7f8c1009d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c17bb2640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c1009de10 con 0x7f8c100b69a0 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c17bb2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8c100b58d0 con 0x7f8c10141670 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c17bb2640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8c10141ff0 con 0x7f8c1009d440 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1594e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8c100b69a0 0x7f8c10144330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49398/0 (socket says 172.21.15.12:49398) 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1594e640 1 -- 172.21.15.12:0/2909564166 learned_addr learned my addr 172.21.15.12:0/2909564166 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1514d640 1 --2- 172.21.15.12:0/2909564166 >> v2:172.21.15.53:3300/0 conn(0x7f8c10141670 0x7f8c10141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1614f640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c1009d440 0x7f8c1009d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3947365805 0 0) 0x7f8c1009de10 con 0x7f8c100b69a0 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bfc005740 con 0x7f8c100b69a0 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1383343031 0 0) 0x7f8bfc005740 con 0x7f8c100b69a0 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c1009d440 msgr2=0x7f8c1009d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c1009d440 0x7f8c1009d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 >> v2:172.21.15.53:3300/0 conn(0x7f8c10141670 msgr2=0x7f8c10141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 --2- 172.21.15.12:0/2909564166 >> v2:172.21.15.53:3300/0 conn(0x7f8c10141670 0x7f8c10141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c100b6d80 con 0x7f8c100b69a0 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f8c00001680 con 0x7f8c100b69a0 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 >> v1:172.21.15.12:6789/0 conn(0x7f8c100b69a0 legacy=0x7f8c10144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c1009d440 0x7f8bfc005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bfc006150 0x7f8bfc006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 0x7f8bfc0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8c100b58d0 con 0x7f8bfc006f60 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10141ff0 con 0x7f8bfc006150 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8bfc00dce0 con 0x7f8c1009d440 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1614f640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 0x7f8bfc0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1594e640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c1009d440 0x7f8bfc005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1614f640 1 -- 172.21.15.12:0/2909564166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c1009d440 msgr2=0x7f8bfc005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1614f640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c1009d440 0x7f8bfc005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1614f640 1 -- 172.21.15.12:0/2909564166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bfc006150 msgr2=0x7f8bfc006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1614f640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bfc006150 0x7f8bfc006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.404+0000 7f8c1614f640 1 -- 172.21.15.12:0/2909564166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc005740 con 0x7f8bfc006f60 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c1614f640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 0x7f8bfc0073e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800ba60 tx=0x7f8c0801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ff0516b35563e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800b600 con 0x7f8bfc006f60 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c080b1070 con 0x7f8bfc006f60 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c1494c640 1 -- 172.21.15.12:0/2909564166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c080ac860 con 0x7f8bfc006f60 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 -- 172.21.15.12:0/2909564166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 msgr2=0x7f8bfc0073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 --2- 172.21.15.12:0/2909564166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 0x7f8bfc0073e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800ba60 tx=0x7f8c0801c680 comp rx=0 tx=0).stop 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c1614f640 1 -- 172.21.15.12:0/2909564166 reap_dead start 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 -- 172.21.15.12:0/2909564166 shutdown_connections 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 -- 172.21.15.12:0/2909564166 >> 172.21.15.12:0/2909564166 conn(0x7f8c100b93b0 msgr2=0x7f8c100b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 -- 172.21.15.12:0/2909564166 shutdown_connections 2024-04-27T07:04:57.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 -- 172.21.15.12:0/2909564166 wait complete. 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 Processor -- start 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 -- start start 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 0x7f8c101cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c100725a0 0x7f8c10070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.408+0000 7f8c17bb2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c100b8e90 0x7f8c1013c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c17bb2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10144de0 con 0x7f8bfc006f60 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c17bb2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10144b00 con 0x7f8c100725a0 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c17bb2640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8c101450c0 con 0x7f8c100b8e90 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1594e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 0x7f8c101cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1594e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 0x7f8c101cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47088/0 (socket says 172.21.15.12:47088) 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1594e640 1 -- 172.21.15.12:0/113195083 learned_addr learned my addr 172.21.15.12:0/113195083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:57.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1614f640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c100b8e90 0x7f8c1013c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1514d640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c100725a0 0x7f8c10070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1594e640 1 -- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c100b8e90 msgr2=0x7f8c1013c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1594e640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c100b8e90 0x7f8c1013c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1594e640 1 -- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c100725a0 msgr2=0x7f8c10070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1594e640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c100725a0 0x7f8c10070540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1594e640 1 -- 172.21.15.12:0/113195083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c100711d0 con 0x7f8bfc006f60 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1594e640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 0x7f8c101cea20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8c00011980 tx=0x7f8c00011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c067fc640 1 -- 172.21.15.12:0/113195083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00010040 con 0x7f8bfc006f60 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c067fc640 1 -- 172.21.15.12:0/113195083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0001a070 con 0x7f8bfc006f60 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c17bb2640 1 -- 172.21.15.12:0/113195083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c10072a80 con 0x7f8bfc006f60 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c067fc640 1 -- 172.21.15.12:0/113195083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00015410 con 0x7f8bfc006f60 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c17bb2640 1 -- 172.21.15.12:0/113195083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c10070a80 con 0x7f8bfc006f60 2024-04-27T07:04:57.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c17bb2640 1 -- 172.21.15.12:0/113195083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c100713a0 con 0x7f8bfc006f60 2024-04-27T07:04:57.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c067fc640 1 -- 172.21.15.12:0/113195083 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8c000155b0 con 0x7f8bfc006f60 2024-04-27T07:04:57.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c067fc640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8bf40683b0 0x7f8bf406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c067fc640 1 -- 172.21.15.12:0/113195083 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f8c00089990 con 0x7f8bfc006f60 2024-04-27T07:04:57.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1514d640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8bf40683b0 0x7f8bf406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.412+0000 7f8c1514d640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8bf40683b0 0x7f8bf406a870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0094c0 tx=0x7f8c0c004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:57.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.416+0000 7f8c067fc640 1 -- 172.21.15.12:0/113195083 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c100713a0 con 0x7f8bfc006f60 2024-04-27T07:04:57.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.632+0000 7f8c17bb2640 1 -- 172.21.15.12:0/113195083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f8c10068c20 con 0x7f8bfc006f60 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.632+0000 7f8c067fc640 1 -- 172.21.15.12:0/113195083 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f8c0005a330 con 0x7f8bfc006f60 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 -- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8bf40683b0 msgr2=0x7f8bf406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8bf40683b0 0x7f8bf406a870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0094c0 tx=0x7f8c0c004ac0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 -- 172.21.15.12:0/113195083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 msgr2=0x7f8c101cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 0x7f8c101cea20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8c00011980 tx=0x7f8c00011e40 comp rx=0 tx=0).stop 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 -- 172.21.15.12:0/113195083 shutdown_connections 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8bf40683b0 0x7f8bf406a870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8c100b8e90 0x7f8c1013c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c100725a0 0x7f8c10070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 --2- 172.21.15.12:0/113195083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bfc006f60 0x7f8c101cea20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 -- 172.21.15.12:0/113195083 >> 172.21.15.12:0/113195083 conn(0x7f8c100b93b0 msgr2=0x7f8c1009d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:57.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 -- 172.21.15.12:0/113195083 shutdown_connections 2024-04-27T07:04:57.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.636+0000 7f8c17bb2640 1 -- 172.21.15.12:0/113195083 wait complete. 2024-04-27T07:04:57.658 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-27T07:04:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f734e583640 1 Processor -- start 2024-04-27T07:04:57.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f734e583640 1 -- start start 2024-04-27T07:04:57.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f734e583640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f734806e370 0x7f734813faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f734e583640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f73480b3650 0x7f73480b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f734e583640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73480b4d70 con 0x7f734806d300 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f734e583640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f73480740d0 con 0x7f73480b3650 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f734e583640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f73480b2bf0 con 0x7f734806e370 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f734cb20640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f734806d300 0x7f73480b5e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35844/0 (socket says 172.21.15.12:35844) 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f734cb20640 1 -- 172.21.15.12:0/2999088415 learned_addr learned my addr 172.21.15.12:0/2999088415 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7347fff640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f734806e370 0x7f734813faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f73477fe640 1 --2- 172.21.15.12:0/2999088415 >> v2:172.21.15.53:3300/0 conn(0x7f73480b3650 0x7f73480b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801207475 0 0) 0x7f73480b4d70 con 0x7f734806d300 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7330003530 con 0x7f734806d300 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2942781253 0 0) 0x7f7330003530 con 0x7f734806d300 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f734806e370 msgr2=0x7f734813faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f734806e370 0x7f734813faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 >> v2:172.21.15.53:3300/0 conn(0x7f73480b3650 msgr2=0x7f73480b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 --2- 172.21.15.12:0/2999088415 >> v2:172.21.15.53:3300/0 conn(0x7f73480b3650 0x7f73480b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7348073e60 con 0x7f734806d300 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f7340001a00 con 0x7f734806d300 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 >> v1:172.21.15.12:6789/0 conn(0x7f734806d300 legacy=0x7f73480b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300039c0 0x7f7330003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330004810 0x7f7330004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f734806e370 0x7f73300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73480740d0 con 0x7f73300039c0 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73480b2bf0 con 0x7f7330004810 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f733000b9d0 con 0x7f734806e370 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f7347fff640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300039c0 0x7f7330003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.772+0000 7f73477fe640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330004810 0x7f7330004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f734cb20640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f734806e370 0x7f73300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f7347fff640 1 -- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f734806e370 msgr2=0x7f73300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f7347fff640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f734806e370 0x7f73300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f7347fff640 1 -- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330004810 msgr2=0x7f7330004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:57.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f7347fff640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330004810 0x7f7330004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f7347fff640 1 -- 172.21.15.12:0/2999088415 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7330003530 con 0x7f73300039c0 2024-04-27T07:04:57.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f73477fe640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7330004810 0x7f7330004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:04:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f7347fff640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300039c0 0x7f7330003de0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7338009b10 tx=0x7f733801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b077a2b23adf074 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73380096b0 con 0x7f73300039c0 2024-04-27T07:04:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73380a6420 con 0x7f73300039c0 2024-04-27T07:04:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f7346ffd640 1 -- 172.21.15.12:0/2999088415 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73380a6810 con 0x7f73300039c0 2024-04-27T07:04:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f734e583640 1 -- 172.21.15.12:0/2999088415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300039c0 msgr2=0x7f7330003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f734e583640 1 --2- 172.21.15.12:0/2999088415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73300039c0 0x7f7330003de0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7338009b10 tx=0x7f733801ca70 comp rx=0 tx=0).stop 2024-04-27T07:04:57.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f734cb20640 1 -- 172.21.15.12:0/2999088415 reap_dead start 2024-04-27T07:04:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f734e583640 1 -- 172.21.15.12:0/2999088415 shutdown_connections 2024-04-27T07:04:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f734e583640 1 -- 172.21.15.12:0/2999088415 >> 172.21.15.12:0/2999088415 conn(0x7f73480b93b0 msgr2=0x7f73480b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f734e583640 1 -- 172.21.15.12:0/2999088415 shutdown_connections 2024-04-27T07:04:57.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.776+0000 7f734e583640 1 -- 172.21.15.12:0/2999088415 wait complete. 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 Processor -- start 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 -- start start 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73300039c0 0x7f734809d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73481de540 0x7f73481d83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73481de9c0 0x7f73481d88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f734813fc20 con 0x7f73481de9c0 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f734813faa0 con 0x7f73300039c0 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f734813fda0 con 0x7f73481de540 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734cb20640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73481de9c0 0x7f73481d88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734cb20640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73481de9c0 0x7f73481d88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58234/0 (socket says 172.21.15.12:58234) 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734cb20640 1 -- 172.21.15.12:0/3283485357 learned_addr learned my addr 172.21.15.12:0/3283485357 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:57.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f73477fe640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73481de540 0x7f73481d83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f7347fff640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73300039c0 0x7f734809d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734cb20640 1 -- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73481de540 msgr2=0x7f73481d83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734cb20640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73481de540 0x7f73481d83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734cb20640 1 -- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73300039c0 msgr2=0x7f734809d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734cb20640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73300039c0 0x7f734809d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734cb20640 1 -- 172.21.15.12:0/3283485357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7348148280 con 0x7f73481de9c0 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f7347fff640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73300039c0 0x7f734809d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734cb20640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73481de9c0 0x7f73481d88f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7340002240 tx=0x7f734000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f7344ff9640 1 -- 172.21.15.12:0/3283485357 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734000eb90 con 0x7f73481de9c0 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f7344ff9640 1 -- 172.21.15.12:0/3283485357 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7340018070 con 0x7f73481de9c0 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 -- 172.21.15.12:0/3283485357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73481e1b30 con 0x7f73481de9c0 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f7344ff9640 1 -- 172.21.15.12:0/3283485357 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340013420 con 0x7f73481de9c0 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 -- 172.21.15.12:0/3283485357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73481e20f0 con 0x7f73481de9c0 2024-04-27T07:04:57.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f734e583640 1 -- 172.21.15.12:0/3283485357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73481df500 con 0x7f73481de9c0 2024-04-27T07:04:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f7344ff9640 1 -- 172.21.15.12:0/3283485357 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f73400135c0 con 0x7f73481de9c0 2024-04-27T07:04:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f7344ff9640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7334068380 0x7f733406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.780+0000 7f7344ff9640 1 -- 172.21.15.12:0/3283485357 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f73400868e0 con 0x7f73481de9c0 2024-04-27T07:04:57.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.784+0000 7f7347fff640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7334068380 0x7f733406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:57.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.784+0000 7f7347fff640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7334068380 0x7f733406a840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f73380ad000 tx=0x7f73380046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:57.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.788+0000 7f7344ff9640 1 -- 172.21.15.12:0/3283485357 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f73481df500 con 0x7f73481de9c0 2024-04-27T07:04:58.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:57.996+0000 7f734e583640 1 -- 172.21.15.12:0/3283485357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f73480b0060 con 0x7f73481de9c0 2024-04-27T07:04:58.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.000+0000 7f7344ff9640 1 -- 172.21.15.12:0/3283485357 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f7340057280 con 0x7f73481de9c0 2024-04-27T07:04:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 -- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7334068380 msgr2=0x7f733406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7334068380 0x7f733406a840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f73380ad000 tx=0x7f73380046d0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 -- 172.21.15.12:0/3283485357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73481de9c0 msgr2=0x7f73481d88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73481de9c0 0x7f73481d88f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7340002240 tx=0x7f734000e530 comp rx=0 tx=0).stop 2024-04-27T07:04:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 -- 172.21.15.12:0/3283485357 shutdown_connections 2024-04-27T07:04:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7334068380 0x7f733406a840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73481de9c0 0x7f73481d88f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f73481de540 0x7f73481d83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 --2- 172.21.15.12:0/3283485357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73300039c0 0x7f734809d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 -- 172.21.15.12:0/3283485357 >> 172.21.15.12:0/3283485357 conn(0x7f73480b93b0 msgr2=0x7f73480af660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:58.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 -- 172.21.15.12:0/3283485357 shutdown_connections 2024-04-27T07:04:58.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.004+0000 7f734e583640 1 -- 172.21.15.12:0/3283485357 wait complete. 2024-04-27T07:04:58.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:04:58.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.140+0000 7f6eb65c7640 1 Processor -- start 2024-04-27T07:04:58.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.140+0000 7f6eb65c7640 1 -- start start 2024-04-27T07:04:58.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.140+0000 7f6eb65c7640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 0x7f6eb00b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.140+0000 7f6eb65c7640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6eb006da10 0x7f6eb0143570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.140+0000 7f6eb65c7640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eb01448e0 con 0x7f6eb00b3db0 2024-04-27T07:04:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.140+0000 7f6eb65c7640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb0144ef0 con 0x7f6eb006da10 2024-04-27T07:04:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.140+0000 7f6eb65c7640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb00b5490 con 0x7f6eb00b4b70 2024-04-27T07:04:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.144+0000 7f6eb4b64640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6eb006da10 0x7f6eb0143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.144+0000 7f6eaffff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 0x7f6eb00b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.144+0000 7f6eaf7fe640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f6eb00b3db0 0x7f6eb006d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:43138/0 (socket says 172.21.15.12:43138) 2024-04-27T07:04:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.144+0000 7f6eaf7fe640 1 -- 172.21.15.12:0/1706401755 learned_addr learned my addr 172.21.15.12:0/1706401755 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:58.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.144+0000 7f6eaffff640 1 -- 172.21.15.12:0/1706401755 >> v2:172.21.15.53:3300/0 conn(0x7f6eb006da10 msgr2=0x7f6eb0143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.144+0000 7f6eaffff640 1 --2- 172.21.15.12:0/1706401755 >> v2:172.21.15.53:3300/0 conn(0x7f6eb006da10 0x7f6eb0143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.144+0000 7f6eaffff640 1 -- 172.21.15.12:0/1706401755 >> v1:172.21.15.53:6789/0 conn(0x7f6eb00b3db0 legacy=0x7f6eb006d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.144+0000 7f6eaffff640 1 -- 172.21.15.12:0/1706401755 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb0144b80 con 0x7f6eb00b4b70 2024-04-27T07:04:58.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.144+0000 7f6eaffff640 1 --2- 172.21.15.12:0/1706401755 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 0x7f6eb00b4f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000ba50 tx=0x7f6ea001e8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cafb03f71fb38ff server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:58.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaeffd640 1 -- 172.21.15.12:0/1706401755 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea00ad070 con 0x7f6eb00b4b70 2024-04-27T07:04:58.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaeffd640 1 -- 172.21.15.12:0/1706401755 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea00a7930 con 0x7f6eb00b4b70 2024-04-27T07:04:58.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaeffd640 1 -- 172.21.15.12:0/1706401755 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea00a7d00 con 0x7f6eb00b4b70 2024-04-27T07:04:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- 172.21.15.12:0/1706401755 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 msgr2=0x7f6eb00b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 --2- 172.21.15.12:0/1706401755 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 0x7f6eb00b4f50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000ba50 tx=0x7f6ea001e8c0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- 172.21.15.12:0/1706401755 shutdown_connections 2024-04-27T07:04:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 --2- 172.21.15.12:0/1706401755 >> v2:172.21.15.53:3300/0 conn(0x7f6eb006da10 0x7f6eb0143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 --2- 172.21.15.12:0/1706401755 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 0x7f6eb00b4f50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- 172.21.15.12:0/1706401755 >> 172.21.15.12:0/1706401755 conn(0x7f6eb0067d10 msgr2=0x7f6eb0068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- 172.21.15.12:0/1706401755 shutdown_connections 2024-04-27T07:04:58.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- 172.21.15.12:0/1706401755 wait complete. 2024-04-27T07:04:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 Processor -- start 2024-04-27T07:04:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- start start 2024-04-27T07:04:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eb006da10 0x7f6eb00731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eb00b3db0 0x7f6eb0142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 0x7f6eb0071250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb0150e70 con 0x7f6eb00b3db0 2024-04-27T07:04:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb0150cf0 con 0x7f6eb006da10 2024-04-27T07:04:58.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb0150ff0 con 0x7f6eb00b4b70 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eb00b3db0 0x7f6eb0142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eb00b3db0 0x7f6eb0142600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58244/0 (socket says 172.21.15.12:58244) 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaf7fe640 1 -- 172.21.15.12:0/3321867979 learned_addr learned my addr 172.21.15.12:0/3321867979 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb4b64640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 0x7f6eb0071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaffff640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eb006da10 0x7f6eb00731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaf7fe640 1 -- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 msgr2=0x7f6eb0071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaf7fe640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 0x7f6eb0071250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaf7fe640 1 -- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eb006da10 msgr2=0x7f6eb00731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaf7fe640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eb006da10 0x7f6eb00731f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaf7fe640 1 -- 172.21.15.12:0/3321867979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb00705e0 con 0x7f6eb00b3db0 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eaf7fe640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eb00b3db0 0x7f6eb0142600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400c420 tx=0x7f6ea400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eacff9640 1 -- 172.21.15.12:0/3321867979 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4017070 con 0x7f6eb00b3db0 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eacff9640 1 -- 172.21.15.12:0/3321867979 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea4005cf0 con 0x7f6eb00b3db0 2024-04-27T07:04:58.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- 172.21.15.12:0/3321867979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb00b35f0 con 0x7f6eb00b3db0 2024-04-27T07:04:58.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eacff9640 1 -- 172.21.15.12:0/3321867979 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40072f0 con 0x7f6eb00b3db0 2024-04-27T07:04:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.148+0000 7f6eb65c7640 1 -- 172.21.15.12:0/3321867979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb014e6c0 con 0x7f6eb00b3db0 2024-04-27T07:04:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.152+0000 7f6eb65c7640 1 -- 172.21.15.12:0/3321867979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb0073770 con 0x7f6eb00b3db0 2024-04-27T07:04:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.152+0000 7f6eacff9640 1 -- 172.21.15.12:0/3321867979 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6ea401a070 con 0x7f6eb00b3db0 2024-04-27T07:04:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.152+0000 7f6eacff9640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6e9c068380 0x7f6e9c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.152+0000 7f6eacff9640 1 -- 172.21.15.12:0/3321867979 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f6ea4085970 con 0x7f6eb00b3db0 2024-04-27T07:04:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.152+0000 7f6eaffff640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6e9c068380 0x7f6e9c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.152+0000 7f6eaffff640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6e9c068380 0x7f6e9c06a840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00b2000 tx=0x7f6ea00a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.156+0000 7f6eacff9640 1 -- 172.21.15.12:0/3321867979 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6eb0073770 con 0x7f6eb00b3db0 2024-04-27T07:04:58.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.320+0000 7f6eb65c7640 1 -- 172.21.15.12:0/3321867979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6eb0068c20 con 0x7f6eb00b3db0 2024-04-27T07:04:58.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eacff9640 1 -- 172.21.15.12:0/3321867979 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+12831 (secure 0 0 0) 0x7f6ea4055c30 con 0x7f6eb00b3db0 2024-04-27T07:04:58.329 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:58.329 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:04:55.370564+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:04:58.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 -- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6e9c068380 msgr2=0x7f6e9c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6e9c068380 0x7f6e9c06a840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00b2000 tx=0x7f6ea00a9070 comp rx=0 tx=0).stop 2024-04-27T07:04:58.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 -- 172.21.15.12:0/3321867979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eb00b3db0 msgr2=0x7f6eb0142600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eb00b3db0 0x7f6eb0142600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400c420 tx=0x7f6ea400c8e0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 -- 172.21.15.12:0/3321867979 shutdown_connections 2024-04-27T07:04:58.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6e9c068380 0x7f6e9c06a840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6eb00b4b70 0x7f6eb0071250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6eb00b3db0 0x7f6eb0142600 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 --2- 172.21.15.12:0/3321867979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6eb006da10 0x7f6eb00731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 -- 172.21.15.12:0/3321867979 >> 172.21.15.12:0/3321867979 conn(0x7f6eb0067d10 msgr2=0x7f6eb00b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 -- 172.21.15.12:0/3321867979 shutdown_connections 2024-04-27T07:04:58.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.324+0000 7f6eb65c7640 1 -- 172.21.15.12:0/3321867979 wait complete. 2024-04-27T07:04:58.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:04:58.348 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:04:58.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd44a9b9640 1 Processor -- start 2024-04-27T07:04:58.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd44a9b9640 1 -- start start 2024-04-27T07:04:58.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd44a9b9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fd4440b3650 0x7fd4440b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd44a9b9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd44406d300 0x7fd4440b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd44a9b9640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4440b4d70 con 0x7fd44406e370 2024-04-27T07:04:58.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd44a9b9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fd4440b5c10 con 0x7fd44406d300 2024-04-27T07:04:58.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd44a9b9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fd4440740d0 con 0x7fd4440b3650 2024-04-27T07:04:58.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd448f56640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd44406d300 0x7fd4440b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd448f56640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd44406d300 0x7fd4440b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58256/0 (socket says 172.21.15.12:58256) 2024-04-27T07:04:58.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd448f56640 1 -- 172.21.15.12:0/788531138 learned_addr learned my addr 172.21.15.12:0/788531138 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:58.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd43bfff640 1 --2- 172.21.15.12:0/788531138 >> v2:172.21.15.53:3300/0 conn(0x7fd4440b3650 0x7fd4440b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd448f56640 1 -- 172.21.15.12:0/788531138 >> v2:172.21.15.53:3300/0 conn(0x7fd4440b3650 msgr2=0x7fd4440b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:58.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd448f56640 1 --2- 172.21.15.12:0/788531138 >> v2:172.21.15.53:3300/0 conn(0x7fd4440b3650 0x7fd4440b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd448f56640 1 -- 172.21.15.12:0/788531138 >> v1:172.21.15.53:6789/0 conn(0x7fd44406e370 legacy=0x7fd44413faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:58.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.468+0000 7fd448f56640 1 -- 172.21.15.12:0/788531138 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd444073e60 con 0x7fd44406d300 2024-04-27T07:04:58.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd448f56640 1 --2- 172.21.15.12:0/788531138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406d300 0x7fd4440b56d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd444068760 tx=0x7fd43c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f3c60b56d1a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:58.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd4437fe640 1 -- 172.21.15.12:0/788531138 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0b0070 con 0x7fd44406d300 2024-04-27T07:04:58.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd4437fe640 1 -- 172.21.15.12:0/788531138 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43c0ab600 con 0x7fd44406d300 2024-04-27T07:04:58.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd4437fe640 1 -- 172.21.15.12:0/788531138 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0ab9d0 con 0x7fd44406d300 2024-04-27T07:04:58.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- 172.21.15.12:0/788531138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406d300 msgr2=0x7fd4440b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 --2- 172.21.15.12:0/788531138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406d300 0x7fd4440b56d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd444068760 tx=0x7fd43c01c450 comp rx=0 tx=0).stop 2024-04-27T07:04:58.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- 172.21.15.12:0/788531138 shutdown_connections 2024-04-27T07:04:58.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 --2- 172.21.15.12:0/788531138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406d300 0x7fd4440b56d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 --2- 172.21.15.12:0/788531138 >> v2:172.21.15.53:3300/0 conn(0x7fd4440b3650 0x7fd4440b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- 172.21.15.12:0/788531138 >> 172.21.15.12:0/788531138 conn(0x7fd4440b93b0 msgr2=0x7fd4440b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:58.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- 172.21.15.12:0/788531138 shutdown_connections 2024-04-27T07:04:58.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- 172.21.15.12:0/788531138 wait complete. 2024-04-27T07:04:58.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 Processor -- start 2024-04-27T07:04:58.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- start start 2024-04-27T07:04:58.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd44406d300 0x7fd44409d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406e370 0x7fd4441e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4440b3650 0x7fd444139460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd444152c40 con 0x7fd44406e370 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441401b0 con 0x7fd44406d300 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fd444152dc0 con 0x7fd4440b3650 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd43bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406e370 0x7fd4441e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd43bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406e370 0x7fd4441e03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58258/0 (socket says 172.21.15.12:58258) 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd43bfff640 1 -- 172.21.15.12:0/71410247 learned_addr learned my addr 172.21.15.12:0/71410247 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd448f56640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4440b3650 0x7fd444139460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd443fff640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd44406d300 0x7fd44409d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd43bfff640 1 -- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4440b3650 msgr2=0x7fd444139460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd43bfff640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4440b3650 0x7fd444139460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd43bfff640 1 -- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd44406d300 msgr2=0x7fd44409d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd43bfff640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd44406d300 0x7fd44409d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd43bfff640 1 -- 172.21.15.12:0/71410247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4440b09e0 con 0x7fd44406e370 2024-04-27T07:04:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd43bfff640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406e370 0x7fd4441e03a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd42c006e90 tx=0x7fd42c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd4417fa640 1 -- 172.21.15.12:0/71410247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00b6c0 con 0x7fd44406e370 2024-04-27T07:04:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd4417fa640 1 -- 172.21.15.12:0/71410247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42c019070 con 0x7fd44406e370 2024-04-27T07:04:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- 172.21.15.12:0/71410247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd444147210 con 0x7fd44406e370 2024-04-27T07:04:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd4417fa640 1 -- 172.21.15.12:0/71410247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00bac0 con 0x7fd44406e370 2024-04-27T07:04:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.472+0000 7fd44a9b9640 1 -- 172.21.15.12:0/71410247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd444150130 con 0x7fd44406e370 2024-04-27T07:04:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.476+0000 7fd44a9b9640 1 -- 172.21.15.12:0/71410247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd444068c20 con 0x7fd44406e370 2024-04-27T07:04:58.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.476+0000 7fd4417fa640 1 -- 172.21.15.12:0/71410247 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd42c00bc60 con 0x7fd44406e370 2024-04-27T07:04:58.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.476+0000 7fd4417fa640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd4240683b0 0x7fd42406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.476+0000 7fd4417fa640 1 -- 172.21.15.12:0/71410247 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fd42c0869f0 con 0x7fd44406e370 2024-04-27T07:04:58.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.476+0000 7fd443fff640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd4240683b0 0x7fd42406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.476+0000 7fd443fff640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd4240683b0 0x7fd42406a870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd434004450 tx=0x7fd434009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.480+0000 7fd4417fa640 1 -- 172.21.15.12:0/71410247 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd42c088e20 con 0x7fd44406e370 2024-04-27T07:04:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.640+0000 7fd44a9b9640 1 -- 172.21.15.12:0/71410247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4440af4f0 con 0x7fd44406e370 2024-04-27T07:04:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.640+0000 7fd4417fa640 1 -- 172.21.15.12:0/71410247 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fd42c05d070 con 0x7fd44406e370 2024-04-27T07:04:58.647 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 -- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd4240683b0 msgr2=0x7fd42406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd4240683b0 0x7fd42406a870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd434004450 tx=0x7fd434009040 comp rx=0 tx=0).stop 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 -- 172.21.15.12:0/71410247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406e370 msgr2=0x7fd4441e03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406e370 0x7fd4441e03a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd42c006e90 tx=0x7fd42c00dc60 comp rx=0 tx=0).stop 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 -- 172.21.15.12:0/71410247 shutdown_connections 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd4240683b0 0x7fd42406a870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fd4440b3650 0x7fd444139460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd44406e370 0x7fd4441e03a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 --2- 172.21.15.12:0/71410247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd44406d300 0x7fd44409d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 -- 172.21.15.12:0/71410247 >> 172.21.15.12:0/71410247 conn(0x7fd4440b93b0 msgr2=0x7fd44413d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 -- 172.21.15.12:0/71410247 shutdown_connections 2024-04-27T07:04:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.644+0000 7fd44a9b9640 1 -- 172.21.15.12:0/71410247 wait complete. 2024-04-27T07:04:58.666 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-27T07:04:58.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.784+0000 7f3bc81e4640 1 Processor -- start 2024-04-27T07:04:58.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.784+0000 7f3bc81e4640 1 -- start start 2024-04-27T07:04:58.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc81e4640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3bc013eef0 0x7f3bc01502c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc81e4640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bc00b5bc0 con 0x7f3bc00b9a70 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc81e4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bc00b9e50 con 0x7f3bc00b5760 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc81e4640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3bc009d980 con 0x7f3bc013eef0 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc5f80640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3bc00b9a70 0x7f3bc009ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35854/0 (socket says 172.21.15.12:35854) 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc5f80640 1 -- 172.21.15.12:0/2656899560 learned_addr learned my addr 172.21.15.12:0/2656899560 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc6781640 1 --2- 172.21.15.12:0/2656899560 >> v2:172.21.15.53:3300/0 conn(0x7f3bc013eef0 0x7f3bc01502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825322956 0 0) 0x7f3bc00b5bc0 con 0x7f3bc00b9a70 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ba8003530 con 0x7f3bc00b9a70 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3850023010 0 0) 0x7f3ba8003530 con 0x7f3bc00b9a70 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 >> v2:172.21.15.53:3300/0 conn(0x7f3bc013eef0 msgr2=0x7f3bc01502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 --2- 172.21.15.12:0/2656899560 >> v2:172.21.15.53:3300/0 conn(0x7f3bc013eef0 0x7f3bc01502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 >> v1:172.21.15.53:6789/0 conn(0x7f3bc00b5760 legacy=0x7f3bc013e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc0073bd0 con 0x7f3bc00b9a70 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f3bb0001a00 con 0x7f3bc00b9a70 2024-04-27T07:04:58.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 >> v1:172.21.15.12:6789/0 conn(0x7f3bc00b9a70 legacy=0x7f3bc009ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 --2- 172.21.15.12:0/2656899560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ba80039c0 0x7f3ba8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 --2- 172.21.15.12:0/2656899560 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ba8004810 0x7f3ba8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 --2- 172.21.15.12:0/2656899560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3ba800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba800c140 con 0x7f3ba80051d0 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba800c2f0 con 0x7f3ba80039c0 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba800c4f0 con 0x7f3ba8004810 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc6781640 1 --2- 172.21.15.12:0/2656899560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3ba800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc5f80640 1 --2- 172.21.15.12:0/2656899560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ba80039c0 0x7f3ba8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc6781640 1 -- 172.21.15.12:0/2656899560 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ba8004810 msgr2=0x7f3ba8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc6781640 1 --2- 172.21.15.12:0/2656899560 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ba8004810 0x7f3ba8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc6781640 1 -- 172.21.15.12:0/2656899560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ba80039c0 msgr2=0x7f3ba8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc6781640 1 --2- 172.21.15.12:0/2656899560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ba80039c0 0x7f3ba8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc6781640 1 -- 172.21.15.12:0/2656899560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba8003530 con 0x7f3ba80051d0 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.788+0000 7f3bc6781640 1 -- 172.21.15.12:0/2656899560 reap_dead start 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc6781640 1 --2- 172.21.15.12:0/2656899560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3ba800b9d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009ac0 tx=0x7f3bb801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282f6c7fe23b8985 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8009690 con 0x7f3ba80051d0 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb80a5570 con 0x7f3ba80051d0 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc4f7e640 1 -- 172.21.15.12:0/2656899560 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb80a5940 con 0x7f3ba80051d0 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- 172.21.15.12:0/2656899560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 msgr2=0x7f3ba800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 --2- 172.21.15.12:0/2656899560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3ba800b9d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009ac0 tx=0x7f3bb801ca90 comp rx=0 tx=0).stop 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- 172.21.15.12:0/2656899560 shutdown_connections 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 --2- 172.21.15.12:0/2656899560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3ba800b9d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- 172.21.15.12:0/2656899560 >> 172.21.15.12:0/2656899560 conn(0x7f3bc0067d10 msgr2=0x7f3bc0068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- 172.21.15.12:0/2656899560 shutdown_connections 2024-04-27T07:04:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- 172.21.15.12:0/2656899560 wait complete. 2024-04-27T07:04:58.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 Processor -- start 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- start start 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3bc01346a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3bc00aea40 0x7f3bc00afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc0143400 0x7f3bc01e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc0147f80 con 0x7f3ba80051d0 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc006ee10 con 0x7f3bc0143400 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc0148100 con 0x7f3bc00aea40 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc5f80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3bc01346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc5f80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3bc01346a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58272/0 (socket says 172.21.15.12:58272) 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc5f80640 1 -- 172.21.15.12:0/3783830694 learned_addr learned my addr 172.21.15.12:0/3783830694 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:58.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc6781640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc0143400 0x7f3bc01e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc577f640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3bc00aea40 0x7f3bc00afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc5f80640 1 -- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3bc00aea40 msgr2=0x7f3bc00afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc5f80640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3bc00aea40 0x7f3bc00afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc5f80640 1 -- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc0143400 msgr2=0x7f3bc01e0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc5f80640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc0143400 0x7f3bc01e0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc5f80640 1 -- 172.21.15.12:0/3783830694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc013f790 con 0x7f3ba80051d0 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc6781640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc0143400 0x7f3bc01e0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc5f80640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3bc01346a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0011820 tx=0x7f3bb0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bbe7fc640 1 -- 172.21.15.12:0/3783830694 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0010040 con 0x7f3ba80051d0 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bbe7fc640 1 -- 172.21.15.12:0/3783830694 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb001a070 con 0x7f3ba80051d0 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- 172.21.15.12:0/3783830694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc0142920 con 0x7f3ba80051d0 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bbe7fc640 1 -- 172.21.15.12:0/3783830694 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb000fd40 con 0x7f3ba80051d0 2024-04-27T07:04:58.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.792+0000 7f3bc81e4640 1 -- 172.21.15.12:0/3783830694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc0142ee0 con 0x7f3ba80051d0 2024-04-27T07:04:58.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.796+0000 7f3bc81e4640 1 -- 172.21.15.12:0/3783830694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc0068c20 con 0x7f3ba80051d0 2024-04-27T07:04:58.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.796+0000 7f3bbe7fc640 1 -- 172.21.15.12:0/3783830694 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3bb001f460 con 0x7f3ba80051d0 2024-04-27T07:04:58.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.796+0000 7f3bbe7fc640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3bac0683b0 0x7f3bac06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:58.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.796+0000 7f3bbe7fc640 1 -- 172.21.15.12:0/3783830694 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f3bb0089780 con 0x7f3ba80051d0 2024-04-27T07:04:58.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.796+0000 7f3bc577f640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3bac0683b0 0x7f3bac06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:58.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.796+0000 7f3bc577f640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3bac0683b0 0x7f3bac06a870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3bc013e020 tx=0x7f3bb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:58.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.800+0000 7f3bbe7fc640 1 -- 172.21.15.12:0/3783830694 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3bb005a120 con 0x7f3ba80051d0 2024-04-27T07:04:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 -- 172.21.15.12:0/3783830694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3bc006ddf0 con 0x7f3ba80051d0 2024-04-27T07:04:58.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bbe7fc640 1 -- 172.21.15.12:0/3783830694 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v25) ==== 84+0+65 (secure 0 0 0) 0x7f3bb0059a20 con 0x7f3ba80051d0 2024-04-27T07:04:58.965 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:04:58.965 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2024-04-27T07:04:58.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-27T07:04:58.966 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2024-04-27T07:04:58.966 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2024-04-27T07:04:58.966 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:04:58.966 INFO:teuthology.orchestra.run.smithi012.stdout:] 2024-04-27T07:04:58.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 -- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3bac0683b0 msgr2=0x7f3bac06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3bac0683b0 0x7f3bac06a870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3bc013e020 tx=0x7f3bb4009040 comp rx=0 tx=0).stop 2024-04-27T07:04:58.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 -- 172.21.15.12:0/3783830694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 msgr2=0x7f3bc01346a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3bc01346a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0011820 tx=0x7f3bb0011ce0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 -- 172.21.15.12:0/3783830694 shutdown_connections 2024-04-27T07:04:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3bac0683b0 0x7f3bac06a870 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3bc0143400 0x7f3bc01e0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3bc00aea40 0x7f3bc00afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 --2- 172.21.15.12:0/3783830694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ba80051d0 0x7f3bc01346a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 -- 172.21.15.12:0/3783830694 >> 172.21.15.12:0/3783830694 conn(0x7f3bc0067d10 msgr2=0x7f3bc006d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 -- 172.21.15.12:0/3783830694 shutdown_connections 2024-04-27T07:04:58.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:58.960+0000 7f3bc81e4640 1 -- 172.21.15.12:0/3783830694 wait complete. 2024-04-27T07:04:58.983 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-27T07:04:58.983 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-27T07:04:59.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd0d728640 1 Processor -- start 2024-04-27T07:04:59.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd0d728640 1 -- start start 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd0d728640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7efd0806d500 0x7efd0809e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd0d728640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efd080b4550 0x7efd08152c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd0d728640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd080afcb0 con 0x7efd0806d8e0 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd0d728640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7efd080ae740 con 0x7efd080b4550 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd0d728640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7efd080aead0 con 0x7efd0806d500 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd067fc640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efd0806d8e0 0x7efd080b3e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35862/0 (socket says 172.21.15.12:35862) 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd067fc640 1 -- 172.21.15.12:0/3634815589 learned_addr learned my addr 172.21.15.12:0/3634815589 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd077fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efd080b4550 0x7efd08152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd06ffd640 1 --2- 172.21.15.12:0/3634815589 >> v2:172.21.15.53:3300/0 conn(0x7efd0806d500 0x7efd0809e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd05ffb640 1 -- 172.21.15.12:0/3634815589 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1512405271 0 0) 0x7efd080afcb0 con 0x7efd0806d8e0 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd05ffb640 1 -- 172.21.15.12:0/3634815589 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcf0003530 con 0x7efd0806d8e0 2024-04-27T07:04:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd077fe640 1 -- 172.21.15.12:0/3634815589 >> v2:172.21.15.53:3300/0 conn(0x7efd0806d500 msgr2=0x7efd0809e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:59.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd077fe640 1 --2- 172.21.15.12:0/3634815589 >> v2:172.21.15.53:3300/0 conn(0x7efd0806d500 0x7efd0809e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd077fe640 1 -- 172.21.15.12:0/3634815589 >> v1:172.21.15.12:6789/0 conn(0x7efd0806d8e0 legacy=0x7efd080b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:59.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.096+0000 7efd077fe640 1 -- 172.21.15.12:0/3634815589 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd080740d0 con 0x7efd080b4550 2024-04-27T07:04:59.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd077fe640 1 --2- 172.21.15.12:0/3634815589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 0x7efd08152c90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd080b3b00 tx=0x7efcf801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92c46850ee904f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:59.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd05ffb640 1 -- 172.21.15.12:0/3634815589 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf80b0070 con 0x7efd080b4550 2024-04-27T07:04:59.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd05ffb640 1 -- 172.21.15.12:0/3634815589 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf80ab600 con 0x7efd080b4550 2024-04-27T07:04:59.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd05ffb640 1 -- 172.21.15.12:0/3634815589 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf80ab9d0 con 0x7efd080b4550 2024-04-27T07:04:59.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd0d728640 1 -- 172.21.15.12:0/3634815589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 msgr2=0x7efd08152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:59.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd0d728640 1 --2- 172.21.15.12:0/3634815589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 0x7efd08152c90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd080b3b00 tx=0x7efcf801c3f0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd0d728640 1 -- 172.21.15.12:0/3634815589 shutdown_connections 2024-04-27T07:04:59.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd0d728640 1 --2- 172.21.15.12:0/3634815589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 0x7efd08152c90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd0d728640 1 --2- 172.21.15.12:0/3634815589 >> v2:172.21.15.53:3300/0 conn(0x7efd0806d500 0x7efd0809e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd0d728640 1 -- 172.21.15.12:0/3634815589 >> 172.21.15.12:0/3634815589 conn(0x7efd080b9840 msgr2=0x7efd080b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:59.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd0d728640 1 -- 172.21.15.12:0/3634815589 shutdown_connections 2024-04-27T07:04:59.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd0d728640 1 -- 172.21.15.12:0/3634815589 wait complete. 2024-04-27T07:04:59.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd0d728640 1 Processor -- start 2024-04-27T07:04:59.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.100+0000 7efd0d728640 1 -- start start 2024-04-27T07:04:59.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd0d728640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0806d500 0x7efd081513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd0d728640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efd0806d8e0 0x7efd0814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd0d728640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 0x7efd0814de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd0d728640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efd0809e960 con 0x7efd080b4550 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd0d728640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efd0809e7e0 con 0x7efd0806d500 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd0d728640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7efd080b5b90 con 0x7efd0806d8e0 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd077fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 0x7efd0814de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd077fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 0x7efd0814de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58298/0 (socket says 172.21.15.12:58298) 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd077fe640 1 -- 172.21.15.12:0/3628863509 learned_addr learned my addr 172.21.15.12:0/3628863509 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd067fc640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efd0806d8e0 0x7efd0814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd06ffd640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0806d500 0x7efd081513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd077fe640 1 -- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efd0806d8e0 msgr2=0x7efd0814d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd077fe640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efd0806d8e0 0x7efd0814d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd077fe640 1 -- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0806d500 msgr2=0x7efd081513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd077fe640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0806d500 0x7efd081513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd077fe640 1 -- 172.21.15.12:0/3628863509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0814d160 con 0x7efd080b4550 2024-04-27T07:04:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd077fe640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 0x7efd0814de00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efcf80ae040 tx=0x7efcf8019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efcef7fe640 1 -- 172.21.15.12:0/3628863509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf80b8020 con 0x7efd080b4550 2024-04-27T07:04:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efcef7fe640 1 -- 172.21.15.12:0/3628863509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf80abdd0 con 0x7efd080b4550 2024-04-27T07:04:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efcef7fe640 1 -- 172.21.15.12:0/3628863509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf80b0040 con 0x7efd080b4550 2024-04-27T07:04:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd0d728640 1 -- 172.21.15.12:0/3628863509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd081e4060 con 0x7efd080b4550 2024-04-27T07:04:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd0d728640 1 -- 172.21.15.12:0/3628863509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd081e4620 con 0x7efd080b4550 2024-04-27T07:04:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efd0d728640 1 -- 172.21.15.12:0/3628863509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd08068d00 con 0x7efd080b4550 2024-04-27T07:04:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efcef7fe640 1 -- 172.21.15.12:0/3628863509 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efcf80bb020 con 0x7efd080b4550 2024-04-27T07:04:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efcef7fe640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efcf4068380 0x7efcf406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.104+0000 7efcef7fe640 1 -- 172.21.15.12:0/3628863509 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7efcf81200e0 con 0x7efd080b4550 2024-04-27T07:04:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.108+0000 7efd06ffd640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efcf4068380 0x7efcf406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.108+0000 7efd06ffd640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efcf4068380 0x7efcf406a840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd00009e90 tx=0x7efd0000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:59.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.112+0000 7efcef7fe640 1 -- 172.21.15.12:0/3628863509 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efcf80c2030 con 0x7efd080b4550 2024-04-27T07:04:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.268+0000 7efd0d728640 1 -- 172.21.15.12:0/3628863509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efd0806e960 con 0x7efd080b4550 2024-04-27T07:04:59.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.536+0000 7efcef7fe640 1 -- 172.21.15.12:0/3628863509 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v26) ==== 138+0+0 (secure 0 0 0) 0x7efcf8016070 con 0x7efd080b4550 2024-04-27T07:04:59.542 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 -- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efcf4068380 msgr2=0x7efcf406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efcf4068380 0x7efcf406a840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd00009e90 tx=0x7efd0000b040 comp rx=0 tx=0).stop 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 -- 172.21.15.12:0/3628863509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 msgr2=0x7efd0814de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 0x7efd0814de00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efcf80ae040 tx=0x7efcf8019190 comp rx=0 tx=0).stop 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 -- 172.21.15.12:0/3628863509 shutdown_connections 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7efcf4068380 0x7efcf406a840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd080b4550 0x7efd0814de00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7efd0806d8e0 0x7efd0814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 --2- 172.21.15.12:0/3628863509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd0806d500 0x7efd081513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 -- 172.21.15.12:0/3628863509 >> 172.21.15.12:0/3628863509 conn(0x7efd080b9840 msgr2=0x7efd080705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 -- 172.21.15.12:0/3628863509 shutdown_connections 2024-04-27T07:04:59.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.540+0000 7efd0d728640 1 -- 172.21.15.12:0/3628863509 wait complete. 2024-04-27T07:04:59.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-27T07:04:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0f951640 1 Processor -- start 2024-04-27T07:04:59.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0f951640 1 -- start start 2024-04-27T07:04:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0f951640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6c080b3680 0x7f6c08145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0f951640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f6c081460b0 0x7f6c08150410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0f951640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0806dac0 con 0x7f6c080b9a10 2024-04-27T07:04:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0f951640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f6c0806e890 con 0x7f6c080b3680 2024-04-27T07:04:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0f951640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f6c080b6cd0 con 0x7f6c081460b0 2024-04-27T07:04:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0ceec640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6c080b3680 0x7f6c08145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0ceec640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6c080b3680 0x7f6c08145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58300/0 (socket says 172.21.15.12:58300) 2024-04-27T07:04:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0ceec640 1 -- 172.21.15.12:0/3138370048 learned_addr learned my addr 172.21.15.12:0/3138370048 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:59.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.704+0000 7f6c0deee640 1 --2- 172.21.15.12:0/3138370048 >> v2:172.21.15.53:3300/0 conn(0x7f6c081460b0 0x7f6c08150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0ceec640 1 -- 172.21.15.12:0/3138370048 >> v2:172.21.15.53:3300/0 conn(0x7f6c081460b0 msgr2=0x7f6c08150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:04:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0ceec640 1 --2- 172.21.15.12:0/3138370048 >> v2:172.21.15.53:3300/0 conn(0x7f6c081460b0 0x7f6c08150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0ceec640 1 -- 172.21.15.12:0/3138370048 >> v1:172.21.15.53:6789/0 conn(0x7f6c080b9a10 legacy=0x7f6c080b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:59.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0ceec640 1 -- 172.21.15.12:0/3138370048 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c080b9df0 con 0x7f6c080b3680 2024-04-27T07:04:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0ceec640 1 --2- 172.21.15.12:0/3138370048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c080b3680 0x7f6c08145b70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c0806edf0 tx=0x7f6bf801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f344989bebefbcc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6bfffff640 1 -- 172.21.15.12:0/3138370048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80b0070 con 0x7f6c080b3680 2024-04-27T07:04:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6bfffff640 1 -- 172.21.15.12:0/3138370048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf80ab600 con 0x7f6c080b3680 2024-04-27T07:04:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6bfffff640 1 -- 172.21.15.12:0/3138370048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80ab9d0 con 0x7f6c080b3680 2024-04-27T07:04:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0f951640 1 -- 172.21.15.12:0/3138370048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c080b3680 msgr2=0x7f6c08145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0f951640 1 --2- 172.21.15.12:0/3138370048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c080b3680 0x7f6c08145b70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c0806edf0 tx=0x7f6bf801c400 comp rx=0 tx=0).stop 2024-04-27T07:04:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0f951640 1 -- 172.21.15.12:0/3138370048 shutdown_connections 2024-04-27T07:04:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0f951640 1 --2- 172.21.15.12:0/3138370048 >> v2:172.21.15.53:3300/0 conn(0x7f6c081460b0 0x7f6c08150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0f951640 1 --2- 172.21.15.12:0/3138370048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c080b3680 0x7f6c08145b70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0f951640 1 -- 172.21.15.12:0/3138370048 >> 172.21.15.12:0/3138370048 conn(0x7f6c08067d10 msgr2=0x7f6c08068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:04:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0f951640 1 -- 172.21.15.12:0/3138370048 shutdown_connections 2024-04-27T07:04:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.708+0000 7f6c0f951640 1 -- 172.21.15.12:0/3138370048 wait complete. 2024-04-27T07:04:59.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 Processor -- start 2024-04-27T07:04:59.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 -- start start 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c080b3680 0x7f6c08072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c080b9a10 0x7f6c081e1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c081460b0 0x7f6c081e1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6c08074020 con 0x7f6c081460b0 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6c08073ea0 con 0x7f6c080b9a10 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f6c08153170 con 0x7f6c080b3680 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0deee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c081460b0 0x7f6c081e1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0deee640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c081460b0 0x7f6c081e1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58302/0 (socket says 172.21.15.12:58302) 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0deee640 1 -- 172.21.15.12:0/2569671941 learned_addr learned my addr 172.21.15.12:0/2569671941 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0ceec640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c080b9a10 0x7f6c081e1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0d6ed640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c080b3680 0x7f6c08072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0deee640 1 -- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c080b3680 msgr2=0x7f6c08072860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0deee640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c080b3680 0x7f6c08072860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0deee640 1 -- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c080b9a10 msgr2=0x7f6c081e1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:04:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0deee640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c080b9a10 0x7f6c081e1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:04:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0deee640 1 -- 172.21.15.12:0/2569671941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c081de680 con 0x7f6c081460b0 2024-04-27T07:04:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0deee640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c081460b0 0x7f6c081e1f90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000efc0 tx=0x7f6c0000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6bfdffb640 1 -- 172.21.15.12:0/2569671941 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00018020 con 0x7f6c081460b0 2024-04-27T07:04:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6bfdffb640 1 -- 172.21.15.12:0/2569671941 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c00014480 con 0x7f6c081460b0 2024-04-27T07:04:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 -- 172.21.15.12:0/2569671941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c081de960 con 0x7f6c081460b0 2024-04-27T07:04:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6bfdffb640 1 -- 172.21.15.12:0/2569671941 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00014870 con 0x7f6c081460b0 2024-04-27T07:04:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 -- 172.21.15.12:0/2569671941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c08143500 con 0x7f6c081460b0 2024-04-27T07:04:59.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.712+0000 7f6c0f951640 1 -- 172.21.15.12:0/2569671941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c08068c20 con 0x7f6c081460b0 2024-04-27T07:04:59.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.716+0000 7f6bfdffb640 1 -- 172.21.15.12:0/2569671941 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6c0001b020 con 0x7f6c081460b0 2024-04-27T07:04:59.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.716+0000 7f6bfdffb640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6bec068380 0x7f6bec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:04:59.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.716+0000 7f6bfdffb640 1 -- 172.21.15.12:0/2569671941 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5227+0+0 (secure 0 0 0) 0x7f6c000875a0 con 0x7f6c081460b0 2024-04-27T07:04:59.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.716+0000 7f6c0d6ed640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6bec068380 0x7f6bec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:04:59.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.716+0000 7f6c0d6ed640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6bec068380 0x7f6bec06a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c04001650 tx=0x7f6c04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:04:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.720+0000 7f6bfdffb640 1 -- 172.21.15.12:0/2569671941 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6c00022030 con 0x7f6c081460b0 2024-04-27T07:04:59.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:04:59.884+0000 7f6c0f951640 1 -- 172.21.15.12:0/2569671941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6c0806d7f0 con 0x7f6c081460b0 2024-04-27T07:05:00.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.540+0000 7f6bfdffb640 1 -- 172.21.15.12:0/2569671941 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v27) ==== 170+0+0 (secure 0 0 0) 0x7f6c00010040 con 0x7f6c081460b0 2024-04-27T07:05:00.546 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2024-04-27T07:05:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 -- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6bec068380 msgr2=0x7f6bec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6bec068380 0x7f6bec06a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c04001650 tx=0x7f6c04009040 comp rx=0 tx=0).stop 2024-04-27T07:05:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 -- 172.21.15.12:0/2569671941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c081460b0 msgr2=0x7f6c081e1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c081460b0 0x7f6c081e1f90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000efc0 tx=0x7f6c0000c490 comp rx=0 tx=0).stop 2024-04-27T07:05:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 -- 172.21.15.12:0/2569671941 shutdown_connections 2024-04-27T07:05:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f6bec068380 0x7f6bec06a840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c081460b0 0x7f6c081e1f90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c080b9a10 0x7f6c081e1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 --2- 172.21.15.12:0/2569671941 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f6c080b3680 0x7f6c08072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 -- 172.21.15.12:0/2569671941 >> 172.21.15.12:0/2569671941 conn(0x7f6c08067d10 msgr2=0x7f6c080b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:00.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 -- 172.21.15.12:0/2569671941 shutdown_connections 2024-04-27T07:05:00.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.544+0000 7f6c0f951640 1 -- 172.21.15.12:0/2569671941 wait complete. 2024-04-27T07:05:00.582 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-27T07:05:00.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e682e640 1 Processor -- start 2024-04-27T07:05:00.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e682e640 1 -- start start 2024-04-27T07:05:00.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e682e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f69e006e370 0x7f69e013faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:00.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e682e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e00b4d70 con 0x7f69e006d300 2024-04-27T07:05:00.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e682e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e0073e60 con 0x7f69e00b3650 2024-04-27T07:05:00.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e682e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f69e00740d0 con 0x7f69e006e370 2024-04-27T07:05:00.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e4dcb640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f69e006d300 0x7f69e0150420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35868/0 (socket says 172.21.15.12:35868) 2024-04-27T07:05:00.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e4dcb640 1 -- 172.21.15.12:0/358343574 learned_addr learned my addr 172.21.15.12:0/358343574 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:00.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69dffff640 1 --2- 172.21.15.12:0/358343574 >> v2:172.21.15.53:3300/0 conn(0x7f69e006e370 0x7f69e013faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747250785 0 0) 0x7f69e00b4d70 con 0x7f69e006d300 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69c8003530 con 0x7f69e006d300 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 888516332 0 0) 0x7f69c8003530 con 0x7f69e006d300 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 >> v2:172.21.15.53:3300/0 conn(0x7f69e006e370 msgr2=0x7f69e013faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 --2- 172.21.15.12:0/358343574 >> v2:172.21.15.53:3300/0 conn(0x7f69e006e370 0x7f69e013faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 >> v1:172.21.15.53:6789/0 conn(0x7f69e00b3650 legacy=0x7f69e00b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e00b2bf0 con 0x7f69e006d300 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f69d8001a00 con 0x7f69e006d300 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 >> v1:172.21.15.12:6789/0 conn(0x7f69e006d300 legacy=0x7f69e0150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c80039c0 0x7f69c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f69c8004810 0x7f69c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69e00740d0 con 0x7f69c80051d0 2024-04-27T07:05:00.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f69c800c170 con 0x7f69c80039c0 2024-04-27T07:05:00.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f69c800c370 con 0x7f69c8004810 2024-04-27T07:05:00.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e4dcb640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:00.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69dffff640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c80039c0 0x7f69c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:00.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69df7fe640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f69c8004810 0x7f69c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:00.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e4dcb640 1 -- 172.21.15.12:0/358343574 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f69c8004810 msgr2=0x7f69c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:00.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e4dcb640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f69c8004810 0x7f69c8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e4dcb640 1 -- 172.21.15.12:0/358343574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c80039c0 msgr2=0x7f69c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:00.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e4dcb640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69c80039c0 0x7f69c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.700+0000 7f69e4dcb640 1 -- 172.21.15.12:0/358343574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c8003530 con 0x7f69c80051d0 2024-04-27T07:05:00.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e4dcb640 1 -- 172.21.15.12:0/358343574 reap_dead start 2024-04-27T07:05:00.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e4dcb640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69c800b9d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f69d800a810 tx=0x7f69d801faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e2478cfc0630ef server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:00.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d800a460 con 0x7f69c80051d0 2024-04-27T07:05:00.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d80aa6f0 con 0x7f69c80051d0 2024-04-27T07:05:00.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69deffd640 1 -- 172.21.15.12:0/358343574 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80aaae0 con 0x7f69c80051d0 2024-04-27T07:05:00.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 -- 172.21.15.12:0/358343574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 msgr2=0x7f69c800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:00.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69c800b9d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f69d800a810 tx=0x7f69d801faa0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 -- 172.21.15.12:0/358343574 shutdown_connections 2024-04-27T07:05:00.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 --2- 172.21.15.12:0/358343574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69c800b9d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 -- 172.21.15.12:0/358343574 >> 172.21.15.12:0/358343574 conn(0x7f69e00b93b0 msgr2=0x7f69e00b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:00.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 -- 172.21.15.12:0/358343574 shutdown_connections 2024-04-27T07:05:00.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 -- 172.21.15.12:0/358343574 wait complete. 2024-04-27T07:05:00.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 Processor -- start 2024-04-27T07:05:00.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 -- start start 2024-04-27T07:05:00.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69e00b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:00.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f69e009d740 0x7f69e0144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:00.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69e00b7a60 0x7f69e00b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:00.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69e013faa0 con 0x7f69c80051d0 2024-04-27T07:05:00.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f69e006e750 con 0x7f69e00b7a60 2024-04-27T07:05:00.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69e682e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f69e013fc20 con 0x7f69e009d740 2024-04-27T07:05:00.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69dffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69e00b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:00.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69dffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69e00b8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58312/0 (socket says 172.21.15.12:58312) 2024-04-27T07:05:00.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.704+0000 7f69dffff640 1 -- 172.21.15.12:0/1955848918 learned_addr learned my addr 172.21.15.12:0/1955848918 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:00.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69df7fe640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f69e009d740 0x7f69e0144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:00.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dffff640 1 -- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f69e009d740 msgr2=0x7f69e0144d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:00.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69e4dcb640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69e00b7a60 0x7f69e00b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:00.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dffff640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f69e009d740 0x7f69e0144d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dffff640 1 -- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69e00b7a60 msgr2=0x7f69e00b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:00.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dffff640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69e00b7a60 0x7f69e00b7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:00.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dffff640 1 -- 172.21.15.12:0/1955848918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e00b8420 con 0x7f69c80051d0 2024-04-27T07:05:00.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69df7fe640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f69e009d740 0x7f69e0144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:00.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dffff640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69e00b8b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f69d0011990 tx=0x7f69d0011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:00.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dcff9640 1 -- 172.21.15.12:0/1955848918 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d000fad0 con 0x7f69c80051d0 2024-04-27T07:05:00.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dcff9640 1 -- 172.21.15.12:0/1955848918 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d000fc70 con 0x7f69c80051d0 2024-04-27T07:05:00.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69e682e640 1 -- 172.21.15.12:0/1955848918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e009dcf0 con 0x7f69c80051d0 2024-04-27T07:05:00.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dcff9640 1 -- 172.21.15.12:0/1955848918 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0014440 con 0x7f69c80051d0 2024-04-27T07:05:00.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69e682e640 1 -- 172.21.15.12:0/1955848918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e0145450 con 0x7f69c80051d0 2024-04-27T07:05:00.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69e682e640 1 -- 172.21.15.12:0/1955848918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e00b7690 con 0x7f69c80051d0 2024-04-27T07:05:00.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dcff9640 1 -- 172.21.15.12:0/1955848918 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f69d0012040 con 0x7f69c80051d0 2024-04-27T07:05:00.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dcff9640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f69cc068380 0x7f69cc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:00.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69dcff9640 1 -- 172.21.15.12:0/1955848918 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5609+0+0 (secure 0 0 0) 0x7f69d0087290 con 0x7f69c80051d0 2024-04-27T07:05:00.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.708+0000 7f69df7fe640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f69cc068380 0x7f69cc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:00.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.712+0000 7f69df7fe640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f69cc068380 0x7f69cc06a840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c10 tx=0x7f69d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:00.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.712+0000 7f69dcff9640 1 -- 172.21.15.12:0/1955848918 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69d0019030 con 0x7f69c80051d0 2024-04-27T07:05:00.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:00.912+0000 7f69e682e640 1 -- 172.21.15.12:0/1955848918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f69e0068c20 con 0x7f69c80051d0 2024-04-27T07:05:01.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.560+0000 7f69dcff9640 1 -- 172.21.15.12:0/1955848918 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f69d0057a20 con 0x7f69c80051d0 2024-04-27T07:05:01.567 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 4 and data pool 5 2024-04-27T07:05:01.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.564+0000 7f69e682e640 1 -- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f69cc068380 msgr2=0x7f69cc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:01.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.564+0000 7f69e682e640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f69cc068380 0x7f69cc06a840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c10 tx=0x7f69d4009040 comp rx=0 tx=0).stop 2024-04-27T07:05:01.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.564+0000 7f69e682e640 1 -- 172.21.15.12:0/1955848918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 msgr2=0x7f69e00b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:01.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.564+0000 7f69e682e640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69e00b8b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f69d0011990 tx=0x7f69d0011e50 comp rx=0 tx=0).stop 2024-04-27T07:05:01.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.568+0000 7f69e682e640 1 -- 172.21.15.12:0/1955848918 shutdown_connections 2024-04-27T07:05:01.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.568+0000 7f69e682e640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f69cc068380 0x7f69cc06a840 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.568+0000 7f69e682e640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f69e00b7a60 0x7f69e00b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.568+0000 7f69e682e640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f69e009d740 0x7f69e0144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.568+0000 7f69e682e640 1 --2- 172.21.15.12:0/1955848918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f69c80051d0 0x7f69e00b8b40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.568+0000 7f69e682e640 1 -- 172.21.15.12:0/1955848918 >> 172.21.15.12:0/1955848918 conn(0x7f69e00b93b0 msgr2=0x7f69e0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:01.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.568+0000 7f69e682e640 1 -- 172.21.15.12:0/1955848918 shutdown_connections 2024-04-27T07:05:01.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.568+0000 7f69e682e640 1 -- 172.21.15.12:0/1955848918 wait complete. 2024-04-27T07:05:01.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:05:01.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81bb260640 1 Processor -- start 2024-04-27T07:05:01.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81bb260640 1 -- start start 2024-04-27T07:05:01.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81bb260640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f81b406e6b0 0x7f81b41418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:01.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81bb260640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f81b406cc90 0x7f81b41481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:01.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81bb260640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f81b4141e00 0x7f81b4148700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81bb260640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f81b406df30 con 0x7f81b4141e00 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81bb260640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f81b406ea90 con 0x7f81b406e6b0 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81bb260640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f81b4147cc0 con 0x7f81b406cc90 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b97fd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f81b4141e00 0x7f81b4148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b97fd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f81b4141e00 0x7f81b4148700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58326/0 (socket says 172.21.15.12:58326) 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b97fd640 1 -- 172.21.15.12:0/3666676007 learned_addr learned my addr 172.21.15.12:0/3666676007 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b3fff640 1 --2- 172.21.15.12:0/3666676007 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f81b406cc90 0x7f81b41481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b8ffc640 1 --2- 172.21.15.12:0/3666676007 >> v2:172.21.15.53:3300/0 conn(0x7f81b406e6b0 0x7f81b41418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b97fd640 1 -- 172.21.15.12:0/3666676007 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f81b406cc90 msgr2=0x7f81b41481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b97fd640 1 --2- 172.21.15.12:0/3666676007 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f81b406cc90 0x7f81b41481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b97fd640 1 -- 172.21.15.12:0/3666676007 >> v2:172.21.15.53:3300/0 conn(0x7f81b406e6b0 msgr2=0x7f81b41418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b97fd640 1 --2- 172.21.15.12:0/3666676007 >> v2:172.21.15.53:3300/0 conn(0x7f81b406e6b0 0x7f81b41418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b97fd640 1 -- 172.21.15.12:0/3666676007 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b40b23b0 con 0x7f81b4141e00 2024-04-27T07:05:01.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.728+0000 7f81b8ffc640 1 --2- 172.21.15.12:0/3666676007 >> v2:172.21.15.53:3300/0 conn(0x7f81b406e6b0 0x7f81b41418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b97fd640 1 --2- 172.21.15.12:0/3666676007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b4141e00 0x7f81b4148700 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f81b406f1a0 tx=0x7f81ac01d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448baa324878155c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b37fe640 1 -- 172.21.15.12:0/3666676007 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0a55e0 con 0x7f81b4141e00 2024-04-27T07:05:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b37fe640 1 -- 172.21.15.12:0/3666676007 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ac0a5780 con 0x7f81b4141e00 2024-04-27T07:05:01.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b37fe640 1 -- 172.21.15.12:0/3666676007 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0a5b50 con 0x7f81b4141e00 2024-04-27T07:05:01.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 -- 172.21.15.12:0/3666676007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b4141e00 msgr2=0x7f81b4148700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:01.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 --2- 172.21.15.12:0/3666676007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b4141e00 0x7f81b4148700 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f81b406f1a0 tx=0x7f81ac01d3c0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 -- 172.21.15.12:0/3666676007 shutdown_connections 2024-04-27T07:05:01.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 --2- 172.21.15.12:0/3666676007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b4141e00 0x7f81b4148700 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 --2- 172.21.15.12:0/3666676007 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f81b406cc90 0x7f81b41481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 --2- 172.21.15.12:0/3666676007 >> v2:172.21.15.53:3300/0 conn(0x7f81b406e6b0 0x7f81b41418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 -- 172.21.15.12:0/3666676007 >> 172.21.15.12:0/3666676007 conn(0x7f81b40bb4f0 msgr2=0x7f81b40bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:01.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 -- 172.21.15.12:0/3666676007 shutdown_connections 2024-04-27T07:05:01.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 -- 172.21.15.12:0/3666676007 wait complete. 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 Processor -- start 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 -- start start 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f81b406cc90 0x7f81b40b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b406e6b0 0x7f81b40b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4141e00 0x7f81b413c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f81b414be30 con 0x7f81b406e6b0 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f81b414bcb0 con 0x7f81b4141e00 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81bb260640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f81b414bfb0 con 0x7f81b406cc90 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b406e6b0 0x7f81b40b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b406e6b0 0x7f81b40b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58328/0 (socket says 172.21.15.12:58328) 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b3fff640 1 -- 172.21.15.12:0/3455709923 learned_addr learned my addr 172.21.15.12:0/3455709923 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b97fd640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4141e00 0x7f81b413c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:01.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b8ffc640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f81b406cc90 0x7f81b40b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b3fff640 1 -- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f81b406cc90 msgr2=0x7f81b40b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b3fff640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f81b406cc90 0x7f81b40b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b3fff640 1 -- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4141e00 msgr2=0x7f81b413c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b3fff640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4141e00 0x7f81b413c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.732+0000 7f81b3fff640 1 -- 172.21.15.12:0/3455709923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b40b2e30 con 0x7f81b406e6b0 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81b3fff640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b406e6b0 0x7f81b40b28f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f81a800d830 tx=0x7f81a800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81b17fa640 1 -- 172.21.15.12:0/3455709923 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8018020 con 0x7f81b406e6b0 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81b17fa640 1 -- 172.21.15.12:0/3455709923 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a80096a0 con 0x7f81b406e6b0 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81bb260640 1 -- 172.21.15.12:0/3455709923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b41456e0 con 0x7f81b406e6b0 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81b17fa640 1 -- 172.21.15.12:0/3455709923 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8009a50 con 0x7f81b406e6b0 2024-04-27T07:05:01.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81bb260640 1 -- 172.21.15.12:0/3455709923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b40b43e0 con 0x7f81b406e6b0 2024-04-27T07:05:01.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81bb260640 1 -- 172.21.15.12:0/3455709923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b4069330 con 0x7f81b406e6b0 2024-04-27T07:05:01.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81b17fa640 1 -- 172.21.15.12:0/3455709923 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f81a801b020 con 0x7f81b406e6b0 2024-04-27T07:05:01.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81b17fa640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f81a0068380 0x7f81a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:01.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81b17fa640 1 -- 172.21.15.12:0/3455709923 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f81a8086f80 con 0x7f81b406e6b0 2024-04-27T07:05:01.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81b8ffc640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f81a0068380 0x7f81a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:01.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.736+0000 7f81b8ffc640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f81a0068380 0x7f81a006a840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f81a400a9a0 tx=0x7f81a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:01.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.740+0000 7f81b17fa640 1 -- 172.21.15.12:0/3455709923 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f81a8022020 con 0x7f81b406e6b0 2024-04-27T07:05:01.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.908+0000 7f81bb260640 1 -- 172.21.15.12:0/3455709923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81b40ba080 con 0x7f81b406e6b0 2024-04-27T07:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.908+0000 7f81b17fa640 1 -- 172.21.15.12:0/3455709923 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+16622 (secure 0 0 0) 0x7f81a8057530 con 0x7f81b406e6b0 2024-04-27T07:05:01.915 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:01.915 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:05:00.918110+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:59.279361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:59.893681+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:05:01.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 -- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f81a0068380 msgr2=0x7f81a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:01.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f81a0068380 0x7f81a006a840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f81a400a9a0 tx=0x7f81a4008040 comp rx=0 tx=0).stop 2024-04-27T07:05:01.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 -- 172.21.15.12:0/3455709923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b406e6b0 msgr2=0x7f81b40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:01.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b406e6b0 0x7f81b40b28f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f81a800d830 tx=0x7f81a800dcf0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 -- 172.21.15.12:0/3455709923 shutdown_connections 2024-04-27T07:05:01.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f81a0068380 0x7f81a006a840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81b4141e00 0x7f81b413c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81b406e6b0 0x7f81b40b28f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 --2- 172.21.15.12:0/3455709923 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f81b406cc90 0x7f81b40b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:01.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 -- 172.21.15.12:0/3455709923 >> 172.21.15.12:0/3455709923 conn(0x7f81b40bb4f0 msgr2=0x7f81b40b9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:01.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 -- 172.21.15.12:0/3455709923 shutdown_connections 2024-04-27T07:05:01.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:01.912+0000 7f81bb260640 1 -- 172.21.15.12:0/3455709923 wait complete. 2024-04-27T07:05:01.938 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:05:02.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.048+0000 7f90102e0640 1 Processor -- start 2024-04-27T07:05:02.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.048+0000 7f90102e0640 1 -- start start 2024-04-27T07:05:02.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f90102e0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f900814f410 0x7f9008151800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f90102e0640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900806dac0 con 0x7f90080b9a10 2024-04-27T07:05:02.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f90102e0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900806e890 con 0x7f90080b3680 2024-04-27T07:05:02.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f90102e0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f90080b6cd0 con 0x7f900814f410 2024-04-27T07:05:02.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e07c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f90080b9a10 0x7f90080b65c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35876/0 (socket says 172.21.15.12:35876) 2024-04-27T07:05:02.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e07c640 1 -- 172.21.15.12:0/564326911 learned_addr learned my addr 172.21.15.12:0/564326911 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:02.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e87d640 1 --2- 172.21.15.12:0/564326911 >> v2:172.21.15.53:3300/0 conn(0x7f900814f410 0x7f9008151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3188656084 0 0) 0x7f8ff8001120 con 0x7f90080b9a10 2024-04-27T07:05:02.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ff0003530 con 0x7f90080b9a10 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2867044252 0 0) 0x7f8ff0003530 con 0x7f90080b9a10 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 >> v2:172.21.15.53:3300/0 conn(0x7f900814f410 msgr2=0x7f9008151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 --2- 172.21.15.12:0/564326911 >> v2:172.21.15.53:3300/0 conn(0x7f900814f410 0x7f9008151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 >> v1:172.21.15.53:6789/0 conn(0x7f90080b3680 legacy=0x7f9008145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90080b9df0 con 0x7f90080b9a10 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f8ff8001850 con 0x7f90080b9a10 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 >> v1:172.21.15.12:6789/0 conn(0x7f90080b9a10 legacy=0x7f90080b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 --2- 172.21.15.12:0/564326911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ff0003a10 0x7f8ff0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 --2- 172.21.15.12:0/564326911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ff0004810 0x7f8ff0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 --2- 172.21.15.12:0/564326911 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ff0005220 0x7f8ff000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90080b6cd0 con 0x7f8ff0003a10 2024-04-27T07:05:02.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff000c170 con 0x7f8ff0004810 2024-04-27T07:05:02.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff000c370 con 0x7f8ff0005220 2024-04-27T07:05:02.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e07c640 1 --2- 172.21.15.12:0/564326911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ff0003a10 0x7f8ff0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e07c640 1 -- 172.21.15.12:0/564326911 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ff0005220 msgr2=0x7f8ff000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:02.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e07c640 1 --2- 172.21.15.12:0/564326911 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ff0005220 0x7f8ff000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e07c640 1 -- 172.21.15.12:0/564326911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ff0004810 msgr2=0x7f8ff0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:02.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d87b640 1 --2- 172.21.15.12:0/564326911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ff0004810 0x7f8ff0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e07c640 1 --2- 172.21.15.12:0/564326911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ff0004810 0x7f8ff0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e07c640 1 -- 172.21.15.12:0/564326911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff0003530 con 0x7f8ff0003a10 2024-04-27T07:05:02.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e87d640 1 -- 172.21.15.12:0/564326911 reap_dead start 2024-04-27T07:05:02.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900e07c640 1 --2- 172.21.15.12:0/564326911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ff0003a10 0x7f8ff0003e30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c2f0 tx=0x7f8ff801cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ab848d406ac84f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:02.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8007e20 con 0x7f8ff0003a10 2024-04-27T07:05:02.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff80a9070 con 0x7f8ff0003a10 2024-04-27T07:05:02.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.052+0000 7f900d07a640 1 -- 172.21.15.12:0/564326911 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8005da0 con 0x7f8ff0003a10 2024-04-27T07:05:02.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- 172.21.15.12:0/564326911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ff0003a10 msgr2=0x7f8ff0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 --2- 172.21.15.12:0/564326911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ff0003a10 0x7f8ff0003e30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c2f0 tx=0x7f8ff801cb00 comp rx=0 tx=0).stop 2024-04-27T07:05:02.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- 172.21.15.12:0/564326911 shutdown_connections 2024-04-27T07:05:02.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 --2- 172.21.15.12:0/564326911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ff0003a10 0x7f8ff0003e30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- 172.21.15.12:0/564326911 >> 172.21.15.12:0/564326911 conn(0x7f9008067d10 msgr2=0x7f9008068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:02.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- 172.21.15.12:0/564326911 shutdown_connections 2024-04-27T07:05:02.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- 172.21.15.12:0/564326911 wait complete. 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 Processor -- start 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- start start 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ff0003a10 0x7f90080b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90081470e0 0x7f90081d4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f90081d4a40 0x7f90081e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f900806d480 con 0x7f90081470e0 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f900806d300 con 0x7f8ff0003a10 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f9008073ea0 con 0x7f90081d4a40 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900d87b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90081470e0 0x7f90081d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900d87b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90081470e0 0x7f90081d4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58352/0 (socket says 172.21.15.12:58352) 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900d87b640 1 -- 172.21.15.12:0/3407226884 learned_addr learned my addr 172.21.15.12:0/3407226884 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900e07c640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ff0003a10 0x7f90080b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900e87d640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f90081d4a40 0x7f90081e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900d87b640 1 -- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f90081d4a40 msgr2=0x7f90081e4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900d87b640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f90081d4a40 0x7f90081e4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900d87b640 1 -- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ff0003a10 msgr2=0x7f90080b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900d87b640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ff0003a10 0x7f90080b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900d87b640 1 -- 172.21.15.12:0/3407226884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9008145690 con 0x7f90081470e0 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900e87d640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f90081d4a40 0x7f90081e4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f900d87b640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90081470e0 0x7f90081d4500 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00d770 tx=0x7f8ffc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f9006ffd640 1 -- 172.21.15.12:0/3407226884 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc005270 con 0x7f90081470e0 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f9006ffd640 1 -- 172.21.15.12:0/3407226884 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ffc017070 con 0x7f90081470e0 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- 172.21.15.12:0/3407226884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90080b2720 con 0x7f90081470e0 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f9006ffd640 1 -- 172.21.15.12:0/3407226884 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00b6b0 con 0x7f90081470e0 2024-04-27T07:05:02.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.056+0000 7f90102e0640 1 -- 172.21.15.12:0/3407226884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9008137f50 con 0x7f90081470e0 2024-04-27T07:05:02.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.060+0000 7f90102e0640 1 -- 172.21.15.12:0/3407226884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9008068c20 con 0x7f90081470e0 2024-04-27T07:05:02.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.060+0000 7f9006ffd640 1 -- 172.21.15.12:0/3407226884 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8ffc00b850 con 0x7f90081470e0 2024-04-27T07:05:02.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.060+0000 7f9006ffd640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ff4068380 0x7f8ff406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.060+0000 7f9006ffd640 1 -- 172.21.15.12:0/3407226884 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f8ffc086780 con 0x7f90081470e0 2024-04-27T07:05:02.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.060+0000 7f900e07c640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ff4068380 0x7f8ff406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.060+0000 7f900e07c640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ff4068380 0x7f8ff406a840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80ab000 tx=0x7f8ff80b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:02.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.064+0000 7f9006ffd640 1 -- 172.21.15.12:0/3407226884 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8ffc056dd0 con 0x7f90081470e0 2024-04-27T07:05:02.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.224+0000 7f90102e0640 1 -- 172.21.15.12:0/3407226884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f900806cc30 con 0x7f90081470e0 2024-04-27T07:05:02.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.224+0000 7f9006ffd640 1 -- 172.21.15.12:0/3407226884 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+16622 (secure 0 0 0) 0x7f8ffc0566d0 con 0x7f90081470e0 2024-04-27T07:05:02.231 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:02.231 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:05:00.918110+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:59.279361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:59.893681+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 -- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ff4068380 msgr2=0x7f8ff406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ff4068380 0x7f8ff406a840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80ab000 tx=0x7f8ff80b0040 comp rx=0 tx=0).stop 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 -- 172.21.15.12:0/3407226884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90081470e0 msgr2=0x7f90081d4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90081470e0 0x7f90081d4500 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00d770 tx=0x7f8ffc00dc30 comp rx=0 tx=0).stop 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 -- 172.21.15.12:0/3407226884 shutdown_connections 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8ff4068380 0x7f8ff406a840 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f90081d4a40 0x7f90081e4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90081470e0 0x7f90081d4500 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 --2- 172.21.15.12:0/3407226884 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ff0003a10 0x7f90080b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 -- 172.21.15.12:0/3407226884 >> 172.21.15.12:0/3407226884 conn(0x7f9008067d10 msgr2=0x7f90080b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 -- 172.21.15.12:0/3407226884 shutdown_connections 2024-04-27T07:05:02.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.228+0000 7f90102e0640 1 -- 172.21.15.12:0/3407226884 wait complete. 2024-04-27T07:05:02.249 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-27T07:05:02.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fddf0c45640 1 Processor -- start 2024-04-27T07:05:02.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fddf0c45640 1 -- start start 2024-04-27T07:05:02.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fddf0c45640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fddec14b7d0 0x7fddec1483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fddf0c45640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddec148910 0x7fddec152660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fddf0c45640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddec0b4a30 con 0x7fddec0b4650 2024-04-27T07:05:02.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fddf0c45640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fddec14bc30 con 0x7fddec14b7d0 2024-04-27T07:05:02.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fddf0c45640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fddec14c1d0 con 0x7fddec148910 2024-04-27T07:05:02.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fddea59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fddec0b4650 0x7fddec073a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35888/0 (socket says 172.21.15.12:35888) 2024-04-27T07:05:02.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fdde9d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fddec14b7d0 0x7fddec1483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fddea59c640 1 -- 172.21.15.12:0/189151058 learned_addr learned my addr 172.21.15.12:0/189151058 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fddead9d640 1 --2- 172.21.15.12:0/189151058 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddec148910 0x7fddec152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fdde9d9b640 1 -- 172.21.15.12:0/189151058 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddec148910 msgr2=0x7fddec152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fdde9d9b640 1 --2- 172.21.15.12:0/189151058 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddec148910 0x7fddec152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fdde9d9b640 1 -- 172.21.15.12:0/189151058 >> v1:172.21.15.12:6789/0 conn(0x7fddec0b4650 legacy=0x7fddec073a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fdde9d9b640 1 -- 172.21.15.12:0/189151058 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec14bef0 con 0x7fddec14b7d0 2024-04-27T07:05:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.372+0000 7fdde959a640 1 -- 172.21.15.12:0/189151058 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715749849 0 0) 0x7fddec0b4a30 con 0x7fddec0b4650 2024-04-27T07:05:02.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde9d9b640 1 --2- 172.21.15.12:0/189151058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec14b7d0 0x7fddec1483d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdde00097c0 tx=0x7fdde001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5be31283c36db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:02.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde959a640 1 -- 172.21.15.12:0/189151058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde00b0070 con 0x7fddec14b7d0 2024-04-27T07:05:02.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde959a640 1 -- 172.21.15.12:0/189151058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde00ab600 con 0x7fddec14b7d0 2024-04-27T07:05:02.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde959a640 1 -- 172.21.15.12:0/189151058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde00ab9d0 con 0x7fddec14b7d0 2024-04-27T07:05:02.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 -- 172.21.15.12:0/189151058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec14b7d0 msgr2=0x7fddec1483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 --2- 172.21.15.12:0/189151058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec14b7d0 0x7fddec1483d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdde00097c0 tx=0x7fdde001c410 comp rx=0 tx=0).stop 2024-04-27T07:05:02.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 -- 172.21.15.12:0/189151058 shutdown_connections 2024-04-27T07:05:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 --2- 172.21.15.12:0/189151058 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddec148910 0x7fddec152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 --2- 172.21.15.12:0/189151058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec14b7d0 0x7fddec1483d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 -- 172.21.15.12:0/189151058 >> 172.21.15.12:0/189151058 conn(0x7fddec0b93b0 msgr2=0x7fddec0b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 -- 172.21.15.12:0/189151058 shutdown_connections 2024-04-27T07:05:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 -- 172.21.15.12:0/189151058 wait complete. 2024-04-27T07:05:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 Processor -- start 2024-04-27T07:05:02.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 -- start start 2024-04-27T07:05:02.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec0b4650 0x7fddec13c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec148910 0x7fddec0ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddec14b7d0 0x7fddec1e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fddec155540 con 0x7fddec148910 2024-04-27T07:05:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddec1553c0 con 0x7fddec0b4650 2024-04-27T07:05:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddf0c45640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fddec1556c0 con 0x7fddec14b7d0 2024-04-27T07:05:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec148910 0x7fddec0ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec148910 0x7fddec0ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58372/0 (socket says 172.21.15.12:58372) 2024-04-27T07:05:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde9d9b640 1 -- 172.21.15.12:0/2937792207 learned_addr learned my addr 172.21.15.12:0/2937792207 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddea59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec0b4650 0x7fddec13c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fddead9d640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddec14b7d0 0x7fddec1e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde9d9b640 1 -- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddec14b7d0 msgr2=0x7fddec1e3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde9d9b640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddec14b7d0 0x7fddec1e3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde9d9b640 1 -- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec0b4650 msgr2=0x7fddec13c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde9d9b640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec0b4650 0x7fddec13c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.376+0000 7fdde9d9b640 1 -- 172.21.15.12:0/2937792207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec140c30 con 0x7fddec148910 2024-04-27T07:05:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fdde9d9b640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec148910 0x7fddec0ae3d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdde00ae040 tx=0x7fdde001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fdddaffd640 1 -- 172.21.15.12:0/2937792207 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde00bc070 con 0x7fddec148910 2024-04-27T07:05:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fdddaffd640 1 -- 172.21.15.12:0/2937792207 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde00b1040 con 0x7fddec148910 2024-04-27T07:05:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fddf0c45640 1 -- 172.21.15.12:0/2937792207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddec0af480 con 0x7fddec148910 2024-04-27T07:05:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fdddaffd640 1 -- 172.21.15.12:0/2937792207 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde00b0050 con 0x7fddec148910 2024-04-27T07:05:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fddf0c45640 1 -- 172.21.15.12:0/2937792207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddec0b85e0 con 0x7fddec148910 2024-04-27T07:05:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fddf0c45640 1 -- 172.21.15.12:0/2937792207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddec068c20 con 0x7fddec148910 2024-04-27T07:05:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fdddaffd640 1 -- 172.21.15.12:0/2937792207 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdde00bf070 con 0x7fddec148910 2024-04-27T07:05:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fdddaffd640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fddcc068380 0x7fddcc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fdddaffd640 1 -- 172.21.15.12:0/2937792207 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7fdde01206a0 con 0x7fddec148910 2024-04-27T07:05:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fddea59c640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fddcc068380 0x7fddcc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.380+0000 7fddea59c640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fddcc068380 0x7fddcc06a840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdddc000f70 tx=0x7fdddc007df0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:02.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.384+0000 7fdddaffd640 1 -- 172.21.15.12:0/2937792207 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdde0016090 con 0x7fddec148910 2024-04-27T07:05:02.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:02.588+0000 7fddf0c45640 1 -- 172.21.15.12:0/2937792207 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fddec1457b0 con 0x7fddec148910 2024-04-27T07:05:03.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.568+0000 7fdddaffd640 1 -- 172.21.15.12:0/2937792207 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fdde00f0780 con 0x7fddec148910 2024-04-27T07:05:03.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 -- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fddcc068380 msgr2=0x7fddcc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:03.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fddcc068380 0x7fddcc06a840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdddc000f70 tx=0x7fdddc007df0 comp rx=0 tx=0).stop 2024-04-27T07:05:03.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 -- 172.21.15.12:0/2937792207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec148910 msgr2=0x7fddec0ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:03.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec148910 0x7fddec0ae3d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdde00ae040 tx=0x7fdde001ac50 comp rx=0 tx=0).stop 2024-04-27T07:05:03.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 -- 172.21.15.12:0/2937792207 shutdown_connections 2024-04-27T07:05:03.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fddcc068380 0x7fddcc06a840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:03.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddec14b7d0 0x7fddec1e3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:03.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddec148910 0x7fddec0ae3d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 --2- 172.21.15.12:0/2937792207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddec0b4650 0x7fddec13c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 -- 172.21.15.12:0/2937792207 >> 172.21.15.12:0/2937792207 conn(0x7fddec0b93b0 msgr2=0x7fddec145d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 -- 172.21.15.12:0/2937792207 shutdown_connections 2024-04-27T07:05:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.572+0000 7fddf0c45640 1 -- 172.21.15.12:0/2937792207 wait complete. 2024-04-27T07:05:03.597 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-27T07:05:03.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-27T07:05:03.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4767a1d640 1 Processor -- start 2024-04-27T07:05:03.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4767a1d640 1 -- start start 2024-04-27T07:05:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4767a1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f47600b73a0 0x7f476014f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4767a1d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f476006e400 0x7f47600b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4767a1d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f476006e830 con 0x7f47600bbfd0 2024-04-27T07:05:03.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4767a1d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f47600b6850 con 0x7f47600b73a0 2024-04-27T07:05:03.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4767a1d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f4760146850 con 0x7f476006e400 2024-04-27T07:05:03.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4764fb8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f47600b73a0 0x7f476014f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:03.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4764fb8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f47600b73a0 0x7f476014f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58376/0 (socket says 172.21.15.12:58376) 2024-04-27T07:05:03.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4764fb8640 1 -- 172.21.15.12:0/1911510239 learned_addr learned my addr 172.21.15.12:0/1911510239 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:03.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4765fba640 1 --2- 172.21.15.12:0/1911510239 >> v2:172.21.15.53:3300/0 conn(0x7f476006e400 0x7f47600b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:03.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4764fb8640 1 -- 172.21.15.12:0/1911510239 >> v2:172.21.15.53:3300/0 conn(0x7f476006e400 msgr2=0x7f47600b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:03.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4764fb8640 1 --2- 172.21.15.12:0/1911510239 >> v2:172.21.15.53:3300/0 conn(0x7f476006e400 0x7f47600b6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:03.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4764fb8640 1 -- 172.21.15.12:0/1911510239 >> v1:172.21.15.53:6789/0 conn(0x7f47600bbfd0 legacy=0x7f4760146140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:03.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.724+0000 7f4764fb8640 1 -- 172.21.15.12:0/1911510239 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47600bc3b0 con 0x7f47600b73a0 2024-04-27T07:05:03.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4764fb8640 1 --2- 172.21.15.12:0/1911510239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47600b73a0 0x7f476014f6e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f476006fc80 tx=0x7f475001c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fea83da9b44453b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:03.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4757fff640 1 -- 172.21.15.12:0/1911510239 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47500b0070 con 0x7f47600b73a0 2024-04-27T07:05:03.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4757fff640 1 -- 172.21.15.12:0/1911510239 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47500ab600 con 0x7f47600b73a0 2024-04-27T07:05:03.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4757fff640 1 -- 172.21.15.12:0/1911510239 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47500ab9d0 con 0x7f47600b73a0 2024-04-27T07:05:03.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4767a1d640 1 -- 172.21.15.12:0/1911510239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47600b73a0 msgr2=0x7f476014f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:03.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4767a1d640 1 --2- 172.21.15.12:0/1911510239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47600b73a0 0x7f476014f6e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f476006fc80 tx=0x7f475001c270 comp rx=0 tx=0).stop 2024-04-27T07:05:03.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4767a1d640 1 -- 172.21.15.12:0/1911510239 shutdown_connections 2024-04-27T07:05:03.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4767a1d640 1 --2- 172.21.15.12:0/1911510239 >> v2:172.21.15.53:3300/0 conn(0x7f476006e400 0x7f47600b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:03.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4767a1d640 1 --2- 172.21.15.12:0/1911510239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47600b73a0 0x7f476014f6e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:03.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4767a1d640 1 -- 172.21.15.12:0/1911510239 >> 172.21.15.12:0/1911510239 conn(0x7f4760068040 msgr2=0x7f47600bb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:03.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4767a1d640 1 -- 172.21.15.12:0/1911510239 shutdown_connections 2024-04-27T07:05:03.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4767a1d640 1 -- 172.21.15.12:0/1911510239 wait complete. 2024-04-27T07:05:03.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4767a1d640 1 Processor -- start 2024-04-27T07:05:03.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.728+0000 7f4767a1d640 1 -- start start 2024-04-27T07:05:03.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4767a1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f476006e400 0x7f47601e83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:03.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4767a1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47600bbfd0 0x7f476013fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:03.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4767a1d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f476014d0b0 0x7f47601400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:03.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4767a1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4760159620 con 0x7f476006e400 2024-04-27T07:05:03.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4767a1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f47601594a0 con 0x7f47600bbfd0 2024-04-27T07:05:03.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4767a1d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f47601597a0 con 0x7f476014d0b0 2024-04-27T07:05:03.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f47657b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f476006e400 0x7f47601e83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:03.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f47657b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f476006e400 0x7f47601e83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58380/0 (socket says 172.21.15.12:58380) 2024-04-27T07:05:03.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f47657b9640 1 -- 172.21.15.12:0/3237308678 learned_addr learned my addr 172.21.15.12:0/3237308678 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:03.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4764fb8640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47600bbfd0 0x7f476013fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4765fba640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f476014d0b0 0x7f47601400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f47657b9640 1 -- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f476014d0b0 msgr2=0x7f47601400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f47657b9640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f476014d0b0 0x7f47601400d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f47657b9640 1 -- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47600bbfd0 msgr2=0x7f476013fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f47657b9640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47600bbfd0 0x7f476013fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f47657b9640 1 -- 172.21.15.12:0/3237308678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47601e8a40 con 0x7f476006e400 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4764fb8640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47600bbfd0 0x7f476013fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f47657b9640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f476006e400 0x7f47601e83a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f475800ba80 tx=0x7f475800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4755ffb640 1 -- 172.21.15.12:0/3237308678 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475800cc00 con 0x7f476006e400 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4755ffb640 1 -- 172.21.15.12:0/3237308678 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f475800cda0 con 0x7f476006e400 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4755ffb640 1 -- 172.21.15.12:0/3237308678 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758003bc0 con 0x7f476006e400 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4767a1d640 1 -- 172.21.15.12:0/3237308678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476014a2e0 con 0x7f476006e400 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4767a1d640 1 -- 172.21.15.12:0/3237308678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4760140610 con 0x7f476006e400 2024-04-27T07:05:03.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4767a1d640 1 -- 172.21.15.12:0/3237308678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4760069330 con 0x7f476006e400 2024-04-27T07:05:03.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4755ffb640 1 -- 172.21.15.12:0/3237308678 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4758006d10 con 0x7f476006e400 2024-04-27T07:05:03.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4755ffb640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4744068380 0x7f474406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:03.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4755ffb640 1 -- 172.21.15.12:0/3237308678 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f47580865a0 con 0x7f476006e400 2024-04-27T07:05:03.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.732+0000 7f4764fb8640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4744068380 0x7f474406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:03.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.736+0000 7f4764fb8640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4744068380 0x7f474406a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47500ad000 tx=0x7f4750008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:03.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.736+0000 7f4755ffb640 1 -- 172.21.15.12:0/3237308678 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4758056b90 con 0x7f476006e400 2024-04-27T07:05:03.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:03.940+0000 7f4767a1d640 1 -- 172.21.15.12:0/3237308678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f47601448e0 con 0x7f476006e400 2024-04-27T07:05:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4755ffb640 1 -- 172.21.15.12:0/3237308678 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f4758056490 con 0x7f476006e400 2024-04-27T07:05:04.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4767a1d640 1 -- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4744068380 msgr2=0x7f474406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:04.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4767a1d640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4744068380 0x7f474406a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47500ad000 tx=0x7f4750008040 comp rx=0 tx=0).stop 2024-04-27T07:05:04.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4767a1d640 1 -- 172.21.15.12:0/3237308678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f476006e400 msgr2=0x7f47601e83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4767a1d640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f476006e400 0x7f47601e83a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f475800ba80 tx=0x7f475800bf40 comp rx=0 tx=0).stop 2024-04-27T07:05:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4767a1d640 1 -- 172.21.15.12:0/3237308678 shutdown_connections 2024-04-27T07:05:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4767a1d640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4744068380 0x7f474406a840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4767a1d640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f476014d0b0 0x7f47601400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4767a1d640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47600bbfd0 0x7f476013fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4767a1d640 1 --2- 172.21.15.12:0/3237308678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f476006e400 0x7f47601e83a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.608+0000 7f4767a1d640 1 -- 172.21.15.12:0/3237308678 >> 172.21.15.12:0/3237308678 conn(0x7f4760068040 msgr2=0x7f4760072740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.612+0000 7f4767a1d640 1 -- 172.21.15.12:0/3237308678 shutdown_connections 2024-04-27T07:05:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.612+0000 7f4767a1d640 1 -- 172.21.15.12:0/3237308678 wait complete. 2024-04-27T07:05:04.635 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-27T07:05:04.636 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:05:04.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.756+0000 7f32320fe640 1 Processor -- start 2024-04-27T07:05:04.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.756+0000 7f32320fe640 1 -- start start 2024-04-27T07:05:04.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f32320fe640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c1487b0 0x7f322c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:04.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f32320fe640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f322c13e5b0 0x7f322c13ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:04.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f32320fe640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f322c0b9ec0 con 0x7f322c0b9ae0 2024-04-27T07:05:04.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f32320fe640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f322c09ddb0 con 0x7f322c13e5b0 2024-04-27T07:05:04.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f32320fe640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f322c09e090 con 0x7f322c1487b0 2024-04-27T07:05:04.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322b7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f322c0b9ae0 0x7f322c1480a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35892/0 (socket says 172.21.15.12:35892) 2024-04-27T07:05:04.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322b7fe640 1 -- 172.21.15.12:0/2622711244 learned_addr learned my addr 172.21.15.12:0/2622711244 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322bfff640 1 --2- 172.21.15.12:0/2622711244 >> v2:172.21.15.53:3300/0 conn(0x7f322c13e5b0 0x7f322c13ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322affd640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c1487b0 0x7f322c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076822431 0 0) 0x7f322c0b9ec0 con 0x7f322c0b9ae0 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3218003530 con 0x7f322c0b9ae0 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 465135054 0 0) 0x7f3218003530 con 0x7f322c0b9ae0 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c1487b0 msgr2=0x7f322c073c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c1487b0 0x7f322c073c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 >> v2:172.21.15.53:3300/0 conn(0x7f322c13e5b0 msgr2=0x7f322c13ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 --2- 172.21.15.12:0/2622711244 >> v2:172.21.15.53:3300/0 conn(0x7f322c13e5b0 0x7f322c13ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c13efa0 con 0x7f322c0b9ae0 2024-04-27T07:05:04.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f3224001a00 con 0x7f322c0b9ae0 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 >> v1:172.21.15.12:6789/0 conn(0x7f322c0b9ae0 legacy=0x7f322c1480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c1487b0 0x7f32180039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3218003f40 0x7f3218004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 0x7f32180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f322c09ddb0 con 0x7f3218004d50 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f322c09e090 con 0x7f3218003f40 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f321800b9d0 con 0x7f322c1487b0 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322bfff640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 0x7f32180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322affd640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3218003f40 0x7f3218004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322b7fe640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c1487b0 0x7f32180039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322bfff640 1 -- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c1487b0 msgr2=0x7f32180039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322bfff640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c1487b0 0x7f32180039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322bfff640 1 -- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3218003f40 msgr2=0x7f3218004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322bfff640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3218003f40 0x7f3218004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322bfff640 1 -- 172.21.15.12:0/2622711244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3218003530 con 0x7f3218004d50 2024-04-27T07:05:04.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.760+0000 7f322b7fe640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c1487b0 0x7f32180039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:04.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322bfff640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 0x7f32180051d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f321c009ae0 tx=0x7f321c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd585af0f21f242 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:04.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c0b2070 con 0x7f3218004d50 2024-04-27T07:05:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c0a56a0 con 0x7f3218004d50 2024-04-27T07:05:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322a7fc640 1 -- 172.21.15.12:0/2622711244 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c0096b0 con 0x7f3218004d50 2024-04-27T07:05:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 -- 172.21.15.12:0/2622711244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 msgr2=0x7f32180051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 --2- 172.21.15.12:0/2622711244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 0x7f32180051d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f321c009ae0 tx=0x7f321c01ca50 comp rx=0 tx=0).stop 2024-04-27T07:05:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322bfff640 1 -- 172.21.15.12:0/2622711244 reap_dead start 2024-04-27T07:05:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 -- 172.21.15.12:0/2622711244 shutdown_connections 2024-04-27T07:05:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 -- 172.21.15.12:0/2622711244 >> 172.21.15.12:0/2622711244 conn(0x7f322c067d10 msgr2=0x7f322c068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 -- 172.21.15.12:0/2622711244 shutdown_connections 2024-04-27T07:05:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 -- 172.21.15.12:0/2622711244 wait complete. 2024-04-27T07:05:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 Processor -- start 2024-04-27T07:05:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 -- start start 2024-04-27T07:05:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 0x7f322c0af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c13f400 0x7f322c139890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c13f880 0x7f322c139dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f322c0b4f00 con 0x7f3218004d50 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f322c09db90 con 0x7f322c13f400 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f32320fe640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f322c0b5080 con 0x7f322c13f880 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 0x7f322c0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 0x7f322c0af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58392/0 (socket says 172.21.15.12:58392) 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322b7fe640 1 -- 172.21.15.12:0/4145028276 learned_addr learned my addr 172.21.15.12:0/4145028276 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322affd640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c13f400 0x7f322c139890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322bfff640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c13f880 0x7f322c139dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322b7fe640 1 -- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c13f880 msgr2=0x7f322c139dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322b7fe640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c13f880 0x7f322c139dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322b7fe640 1 -- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c13f400 msgr2=0x7f322c139890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322b7fe640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c13f400 0x7f322c139890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322b7fe640 1 -- 172.21.15.12:0/4145028276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c1414e0 con 0x7f3218004d50 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322bfff640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c13f880 0x7f322c139dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322affd640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c13f400 0x7f322c139890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f322b7fe640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 0x7f322c0af680 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3224001b50 tx=0x7f322400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f3223fff640 1 -- 172.21.15.12:0/4145028276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224019070 con 0x7f3218004d50 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f3223fff640 1 -- 172.21.15.12:0/4145028276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3224004d70 con 0x7f3218004d50 2024-04-27T07:05:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.764+0000 7f3223fff640 1 -- 172.21.15.12:0/4145028276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240072d0 con 0x7f3218004d50 2024-04-27T07:05:04.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.768+0000 7f32320fe640 1 -- 172.21.15.12:0/4145028276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c142040 con 0x7f3218004d50 2024-04-27T07:05:04.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.768+0000 7f32320fe640 1 -- 172.21.15.12:0/4145028276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c13cb20 con 0x7f3218004d50 2024-04-27T07:05:04.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.768+0000 7f32320fe640 1 -- 172.21.15.12:0/4145028276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322c068c20 con 0x7f3218004d50 2024-04-27T07:05:04.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.768+0000 7f3223fff640 1 -- 172.21.15.12:0/4145028276 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3224007470 con 0x7f3218004d50 2024-04-27T07:05:04.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.768+0000 7f3223fff640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f320c068380 0x7f320c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:04.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.768+0000 7f3223fff640 1 -- 172.21.15.12:0/4145028276 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f3224086b80 con 0x7f3218004d50 2024-04-27T07:05:04.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.768+0000 7f322affd640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f320c068380 0x7f320c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:04.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.768+0000 7f322affd640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f320c068380 0x7f320c06a840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f322c13e230 tx=0x7f3214008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:04.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.772+0000 7f3223fff640 1 -- 172.21.15.12:0/4145028276 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3224057290 con 0x7f3218004d50 2024-04-27T07:05:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.968+0000 7f32320fe640 1 -- 172.21.15.12:0/4145028276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f322c0b9ec0 con 0x7f3218004d50 2024-04-27T07:05:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.968+0000 7f3223fff640 1 -- 172.21.15.12:0/4145028276 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3531 (secure 0 0 0) 0x7f3224056b90 con 0x7f3218004d50 2024-04-27T07:05:04.975 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:04.975 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3153112684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3153112684},{"type":"v1","addr":"172.21.15.53:6835","nonce":3153112684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:05:00.918068+0000","modified":"2024-04-27T07:05:03.947324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5001":{"gid":5001,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4067812811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4067812811},{"type":"v1","addr":"172.21.15.12:6833","nonce":4067812811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5001,"qdb_cluster":[5001]},"id":2}]} 2024-04-27T07:05:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 16 2024-04-27T07:05:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 -- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f320c068380 msgr2=0x7f320c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f320c068380 0x7f320c06a840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f322c13e230 tx=0x7f3214008040 comp rx=0 tx=0).stop 2024-04-27T07:05:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 -- 172.21.15.12:0/4145028276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 msgr2=0x7f322c0af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 0x7f322c0af680 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3224001b50 tx=0x7f322400b560 comp rx=0 tx=0).stop 2024-04-27T07:05:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 -- 172.21.15.12:0/4145028276 shutdown_connections 2024-04-27T07:05:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f320c068380 0x7f320c06a840 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f322c13f880 0x7f322c139dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f322c13f400 0x7f322c139890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 --2- 172.21.15.12:0/4145028276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3218004d50 0x7f322c0af680 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 -- 172.21.15.12:0/4145028276 >> 172.21.15.12:0/4145028276 conn(0x7f322c067d10 msgr2=0x7f322c073ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 -- 172.21.15.12:0/4145028276 shutdown_connections 2024-04-27T07:05:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:04.972+0000 7f32320fe640 1 -- 172.21.15.12:0/4145028276 wait complete. 2024-04-27T07:05:04.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:05:05.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.104+0000 7f8baf03e640 1 Processor -- start 2024-04-27T07:05:05.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.104+0000 7f8baf03e640 1 -- start start 2024-04-27T07:05:05.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8baf03e640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8ba80b9ae0 0x7f8ba8073c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8baf03e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 0x7f8ba8148500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8baf03e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba8148a40 con 0x7f8ba813e5b0 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8baf03e640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba80b9ec0 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8baf03e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba809ddb0 con 0x7f8ba81480a0 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8ba7fff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 0x7f8ba8148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8bacdda640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8ba80b9ae0 0x7f8ba8073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8ba7fff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 0x7f8ba8148500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:51394/0 (socket says 172.21.15.12:51394) 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8ba7fff640 1 -- 172.21.15.12:0/4252427768 learned_addr learned my addr 172.21.15.12:0/4252427768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8ba7fff640 1 -- 172.21.15.12:0/4252427768 >> v2:172.21.15.53:3300/0 conn(0x7f8ba80b9ae0 msgr2=0x7f8ba8073c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8ba7fff640 1 --2- 172.21.15.12:0/4252427768 >> v2:172.21.15.53:3300/0 conn(0x7f8ba80b9ae0 0x7f8ba8073c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8ba7fff640 1 -- 172.21.15.12:0/4252427768 >> v1:172.21.15.53:6789/0 conn(0x7f8ba813e5b0 legacy=0x7f8ba813ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8ba7fff640 1 -- 172.21.15.12:0/4252427768 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba809e090 con 0x7f8ba81480a0 2024-04-27T07:05:05.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.108+0000 7f8ba7fff640 1 --2- 172.21.15.12:0/4252427768 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 0x7f8ba8148500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00bb10 tx=0x7f8b9c01ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152adbb18ccfad2c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:05.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8ba77fe640 1 -- 172.21.15.12:0/4252427768 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0b4070 con 0x7f8ba81480a0 2024-04-27T07:05:05.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8ba77fe640 1 -- 172.21.15.12:0/4252427768 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9c0a7a00 con 0x7f8ba81480a0 2024-04-27T07:05:05.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8ba77fe640 1 -- 172.21.15.12:0/4252427768 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0a7dd0 con 0x7f8ba81480a0 2024-04-27T07:05:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- 172.21.15.12:0/4252427768 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 msgr2=0x7f8ba8148500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 --2- 172.21.15.12:0/4252427768 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 0x7f8ba8148500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00bb10 tx=0x7f8b9c01ec30 comp rx=0 tx=0).stop 2024-04-27T07:05:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- 172.21.15.12:0/4252427768 shutdown_connections 2024-04-27T07:05:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 --2- 172.21.15.12:0/4252427768 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 0x7f8ba8148500 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 --2- 172.21.15.12:0/4252427768 >> v2:172.21.15.53:3300/0 conn(0x7f8ba80b9ae0 0x7f8ba8073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- 172.21.15.12:0/4252427768 >> 172.21.15.12:0/4252427768 conn(0x7f8ba8067d10 msgr2=0x7f8ba8068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- 172.21.15.12:0/4252427768 shutdown_connections 2024-04-27T07:05:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- 172.21.15.12:0/4252427768 wait complete. 2024-04-27T07:05:05.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 Processor -- start 2024-04-27T07:05:05.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- start start 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba80b9ae0 0x7f8ba814e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba813e5b0 0x7f8ba80af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 0x7f8ba80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba806dd20 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba809db90 con 0x7f8ba813e5b0 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba806dea0 con 0x7f8ba81480a0 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8bacdda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba80b9ae0 0x7f8ba814e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8bacdda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba80b9ae0 0x7f8ba814e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58398/0 (socket says 172.21.15.12:58398) 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8bacdda640 1 -- 172.21.15.12:0/1734937154 learned_addr learned my addr 172.21.15.12:0/1734937154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8ba7fff640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba813e5b0 0x7f8ba80af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8bad5db640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 0x7f8ba80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8bacdda640 1 -- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 msgr2=0x7f8ba80afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8bacdda640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 0x7f8ba80afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8bacdda640 1 -- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba813e5b0 msgr2=0x7f8ba80af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8bacdda640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba813e5b0 0x7f8ba80af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8bacdda640 1 -- 172.21.15.12:0/1734937154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba814ee50 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8bacdda640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba80b9ae0 0x7f8ba814e4f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800e870 tx=0x7f8b9800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8ba57fa640 1 -- 172.21.15.12:0/1734937154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98017070 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8ba57fa640 1 -- 172.21.15.12:0/1734937154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9800c940 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- 172.21.15.12:0/1734937154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba8147b20 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8ba57fa640 1 -- 172.21.15.12:0/1734937154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9800ccf0 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.112+0000 7f8baf03e640 1 -- 172.21.15.12:0/1734937154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba80af130 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.116+0000 7f8baf03e640 1 -- 172.21.15.12:0/1734937154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba8068c20 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.116+0000 7f8ba57fa640 1 -- 172.21.15.12:0/1734937154 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8b9801a070 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.116+0000 7f8ba57fa640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b940683b0 0x7f8b9406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.116+0000 7f8ba57fa640 1 -- 172.21.15.12:0/1734937154 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f8b980867b0 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.116+0000 7f8ba7fff640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b940683b0 0x7f8b9406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.116+0000 7f8ba7fff640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b940683b0 0x7f8b9406a870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0b1000 tx=0x7f8b9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:05.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.120+0000 7f8ba57fa640 1 -- 172.21.15.12:0/1734937154 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8b98056e00 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.284+0000 7f8baf03e640 1 -- 172.21.15.12:0/1734937154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ba80b6420 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.284+0000 7f8ba57fa640 1 -- 172.21.15.12:0/1734937154 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16609 (secure 0 0 0) 0x7f8b98056700 con 0x7f8ba80b9ae0 2024-04-27T07:05:05.292 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:05.292 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:05:02.565909+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:59.279361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:59.893681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:05:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 -- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b940683b0 msgr2=0x7f8b9406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b940683b0 0x7f8b9406a870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0b1000 tx=0x7f8b9c008040 comp rx=0 tx=0).stop 2024-04-27T07:05:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 -- 172.21.15.12:0/1734937154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba80b9ae0 msgr2=0x7f8ba814e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba80b9ae0 0x7f8ba814e4f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800e870 tx=0x7f8b9800ec70 comp rx=0 tx=0).stop 2024-04-27T07:05:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 -- 172.21.15.12:0/1734937154 shutdown_connections 2024-04-27T07:05:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b940683b0 0x7f8b9406a870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8ba81480a0 0x7f8ba80afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8ba813e5b0 0x7f8ba80af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 --2- 172.21.15.12:0/1734937154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ba80b9ae0 0x7f8ba814e4f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 -- 172.21.15.12:0/1734937154 >> 172.21.15.12:0/1734937154 conn(0x7f8ba8067d10 msgr2=0x7f8ba80b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:05.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 -- 172.21.15.12:0/1734937154 shutdown_connections 2024-04-27T07:05:05.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.288+0000 7f8baf03e640 1 -- 172.21.15.12:0/1734937154 wait complete. 2024-04-27T07:05:05.311 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-27T07:05:05.312 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:05:05.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.424+0000 7f1ba39de640 1 Processor -- start 2024-04-27T07:05:05.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba39de640 1 -- start start 2024-04-27T07:05:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba39de640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1b9c0b64c0 0x7f1b9c0b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba39de640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9c0b7560 con 0x7f1b9c148810 2024-04-27T07:05:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba39de640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9c0b68a0 con 0x7f1b9c0b5ae0 2024-04-27T07:05:05.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba39de640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9c148c80 con 0x7f1b9c0b64c0 2024-04-27T07:05:05.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba1f7b640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1b9c148810 0x7f1b9c152640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35902/0 (socket says 172.21.15.12:35902) 2024-04-27T07:05:05.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba1f7b640 1 -- 172.21.15.12:0/219742568 learned_addr learned my addr 172.21.15.12:0/219742568 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:05.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba177a640 1 --2- 172.21.15.12:0/219742568 >> v2:172.21.15.53:3300/0 conn(0x7f1b9c0b64c0 0x7f1b9c0b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1395908056 0 0) 0x7f1b9c0b7560 con 0x7f1b9c148810 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b88003530 con 0x7f1b9c148810 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1638656422 0 0) 0x7f1b88003530 con 0x7f1b9c148810 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 >> v2:172.21.15.53:3300/0 conn(0x7f1b9c0b64c0 msgr2=0x7f1b9c0b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 --2- 172.21.15.12:0/219742568 >> v2:172.21.15.53:3300/0 conn(0x7f1b9c0b64c0 0x7f1b9c0b7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 >> v1:172.21.15.53:6789/0 conn(0x7f1b9c0b5ae0 legacy=0x7f1b9c141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c148010 con 0x7f1b9c148810 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f1b94001a20 con 0x7f1b9c148810 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 >> v1:172.21.15.12:6789/0 conn(0x7f1b9c148810 legacy=0x7f1b9c152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b880039c0 0x7f1b88003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b88004810 0x7f1b88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b880052f0 0x7f1b8800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c148c80 con 0x7f1b880039c0 2024-04-27T07:05:05.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8800c1c0 con 0x7f1b880052f0 2024-04-27T07:05:05.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8800c3c0 con 0x7f1b88004810 2024-04-27T07:05:05.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba177a640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b880039c0 0x7f1b88003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba0f79640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b88004810 0x7f1b88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba1f7b640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b880052f0 0x7f1b8800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba177a640 1 -- 172.21.15.12:0/219742568 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b88004810 msgr2=0x7f1b88004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:05.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba177a640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b88004810 0x7f1b88004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba177a640 1 -- 172.21.15.12:0/219742568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b880052f0 msgr2=0x7f1b8800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba177a640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b880052f0 0x7f1b8800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba177a640 1 -- 172.21.15.12:0/219742568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b88003530 con 0x7f1b880039c0 2024-04-27T07:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba1f7b640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b880052f0 0x7f1b8800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba1f7b640 1 -- 172.21.15.12:0/219742568 reap_dead start 2024-04-27T07:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.428+0000 7f1ba0f79640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b88004810 0x7f1b88004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:05.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba177a640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b880039c0 0x7f1b88003de0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009ab0 tx=0x7f1b9801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d08932b87e32d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:05.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b980096b0 con 0x7f1b880039c0 2024-04-27T07:05:05.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b980a6430 con 0x7f1b880039c0 2024-04-27T07:05:05.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1b93fff640 1 -- 172.21.15.12:0/219742568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b980a6820 con 0x7f1b880039c0 2024-04-27T07:05:05.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 -- 172.21.15.12:0/219742568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b880039c0 msgr2=0x7f1b88003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b880039c0 0x7f1b88003de0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009ab0 tx=0x7f1b9801ca50 comp rx=0 tx=0).stop 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 -- 172.21.15.12:0/219742568 shutdown_connections 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 --2- 172.21.15.12:0/219742568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b880039c0 0x7f1b88003de0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 -- 172.21.15.12:0/219742568 >> 172.21.15.12:0/219742568 conn(0x7f1b9c073c00 msgr2=0x7f1b9c0bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 -- 172.21.15.12:0/219742568 shutdown_connections 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 -- 172.21.15.12:0/219742568 wait complete. 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 Processor -- start 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 -- start start 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b880039c0 0x7f1b9c13f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9c14adc0 0x7f1b9c143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b9c144e30 0x7f1b9c135930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c14b2b0 con 0x7f1b9c144e30 2024-04-27T07:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c14b430 con 0x7f1b9c14adc0 2024-04-27T07:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba39de640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9c14b5b0 con 0x7f1b880039c0 2024-04-27T07:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba1f7b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b9c144e30 0x7f1b9c135930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba1f7b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b9c144e30 0x7f1b9c135930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58410/0 (socket says 172.21.15.12:58410) 2024-04-27T07:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba1f7b640 1 -- 172.21.15.12:0/637618019 learned_addr learned my addr 172.21.15.12:0/637618019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.432+0000 7f1ba0f79640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9c14adc0 0x7f1b9c143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba177a640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b880039c0 0x7f1b9c13f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba1f7b640 1 -- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b880039c0 msgr2=0x7f1b9c13f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba1f7b640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b880039c0 0x7f1b9c13f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba1f7b640 1 -- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9c14adc0 msgr2=0x7f1b9c143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba1f7b640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9c14adc0 0x7f1b9c143900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba1f7b640 1 -- 172.21.15.12:0/637618019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c143e40 con 0x7f1b9c144e30 2024-04-27T07:05:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba0f79640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9c14adc0 0x7f1b9c143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba1f7b640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b9c144e30 0x7f1b9c135930 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0bb150 tx=0x7f1b9400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1b91ffb640 1 -- 172.21.15.12:0/637618019 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94004d70 con 0x7f1b9c144e30 2024-04-27T07:05:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1b91ffb640 1 -- 172.21.15.12:0/637618019 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b9400bb80 con 0x7f1b9c144e30 2024-04-27T07:05:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba39de640 1 -- 172.21.15.12:0/637618019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c144110 con 0x7f1b9c144e30 2024-04-27T07:05:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1b91ffb640 1 -- 172.21.15.12:0/637618019 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94007110 con 0x7f1b9c144e30 2024-04-27T07:05:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba39de640 1 -- 172.21.15.12:0/637618019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c14aac0 con 0x7f1b9c144e30 2024-04-27T07:05:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba39de640 1 -- 172.21.15.12:0/637618019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9c068c20 con 0x7f1b9c144e30 2024-04-27T07:05:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1b91ffb640 1 -- 172.21.15.12:0/637618019 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1b9401b020 con 0x7f1b9c144e30 2024-04-27T07:05:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1b91ffb640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1b80068380 0x7f1b8006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1b91ffb640 1 -- 172.21.15.12:0/637618019 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f1b94086ad0 con 0x7f1b9c144e30 2024-04-27T07:05:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba177a640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1b80068380 0x7f1b8006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.436+0000 7f1ba177a640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1b80068380 0x7f1b8006a840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1b980ad000 tx=0x7f1b980046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:05.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.440+0000 7f1b91ffb640 1 -- 172.21.15.12:0/637618019 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b94057080 con 0x7f1b9c144e30 2024-04-27T07:05:05.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.600+0000 7f1ba39de640 1 -- 172.21.15.12:0/637618019 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1b9c1425b0 con 0x7f1b80068380 2024-04-27T07:05:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.624+0000 7f1b91ffb640 1 -- 172.21.15.12:0/637618019 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251954 (secure 0 0 0) 0x7f1b9c1425b0 con 0x7f1b80068380 2024-04-27T07:05:05.634 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:05.636 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 -- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1b80068380 msgr2=0x7f1b8006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1b80068380 0x7f1b8006a840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1b980ad000 tx=0x7f1b980046d0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 -- 172.21.15.12:0/637618019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b9c144e30 msgr2=0x7f1b9c135930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b9c144e30 0x7f1b9c135930 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0bb150 tx=0x7f1b9400b4a0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 -- 172.21.15.12:0/637618019 shutdown_connections 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1b80068380 0x7f1b8006a840 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b9c144e30 0x7f1b9c135930 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1b9c14adc0 0x7f1b9c143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 --2- 172.21.15.12:0/637618019 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1b880039c0 0x7f1b9c13f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 -- 172.21.15.12:0/637618019 >> 172.21.15.12:0/637618019 conn(0x7f1b9c073c00 msgr2=0x7f1b9c13eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 -- 172.21.15.12:0/637618019 shutdown_connections 2024-04-27T07:05:05.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.628+0000 7f1ba39de640 1 -- 172.21.15.12:0/637618019 wait complete. 2024-04-27T07:05:05.652 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-04-27T07:05:04.625671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236568,"kb_used_data":3740,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738152,"statfs":{"total":773094113280,"available":772851867648,"internally_reserved":0,"allocated":3829760,"data_stored":2171352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.083351"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577259+0000","last_change":"2024-04-27T07:05:01.619766+0000","last_active":"2024-04-27T07:05:02.577259+0000","last_peered":"2024-04-27T07:05:02.577259+0000","last_clean":"2024-04-27T07:05:02.577259+0000","last_became_active":"2024-04-27T07:05:01.619106+0000","last_became_peered":"2024-04-27T07:05:01.619106+0000","last_unstale":"2024-04-27T07:05:02.577259+0000","last_undegraded":"2024-04-27T07:05:02.577259+0000","last_fullsized":"2024-04-27T07:05:02.577259+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:56:25.278857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903942+0000","last_change":"2024-04-27T07:05:01.590887+0000","last_active":"2024-04-27T07:05:02.903942+0000","last_peered":"2024-04-27T07:05:02.903942+0000","last_clean":"2024-04-27T07:05:02.903942+0000","last_became_active":"2024-04-27T07:05:01.590574+0000","last_became_peered":"2024-04-27T07:05:01.590574+0000","last_unstale":"2024-04-27T07:05:02.903942+0000","last_undegraded":"2024-04-27T07:05:02.903942+0000","last_fullsized":"2024-04-27T07:05:02.903942+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:11.729329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904326+0000","last_change":"2024-04-27T07:05:01.600366+0000","last_active":"2024-04-27T07:05:02.904326+0000","last_peered":"2024-04-27T07:05:02.904326+0000","last_clean":"2024-04-27T07:05:02.904326+0000","last_became_active":"2024-04-27T07:05:01.599448+0000","last_became_peered":"2024-04-27T07:05:01.599448+0000","last_unstale":"2024-04-27T07:05:02.904326+0000","last_undegraded":"2024-04-27T07:05:02.904326+0000","last_fullsized":"2024-04-27T07:05:02.904326+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:22:36.544610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905027+0000","last_change":"2024-04-27T07:05:01.601089+0000","last_active":"2024-04-27T07:05:02.905027+0000","last_peered":"2024-04-27T07:05:02.905027+0000","last_clean":"2024-04-27T07:05:02.905027+0000","last_became_active":"2024-04-27T07:05:01.600833+0000","last_became_peered":"2024-04-27T07:05:01.600833+0000","last_unstale":"2024-04-27T07:05:02.905027+0000","last_undegraded":"2024-04-27T07:05:02.905027+0000","last_fullsized":"2024-04-27T07:05:02.905027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:07:11.751089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576494+0000","last_change":"2024-04-27T07:05:01.596148+0000","last_active":"2024-04-27T07:05:02.576494+0000","last_peered":"2024-04-27T07:05:02.576494+0000","last_clean":"2024-04-27T07:05:02.576494+0000","last_became_active":"2024-04-27T07:05:01.595610+0000","last_became_peered":"2024-04-27T07:05:01.595610+0000","last_unstale":"2024-04-27T07:05:02.576494+0000","last_undegraded":"2024-04-27T07:05:02.576494+0000","last_fullsized":"2024-04-27T07:05:02.576494+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:35:42.870388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903963+0000","last_change":"2024-04-27T07:05:01.612863+0000","last_active":"2024-04-27T07:05:02.903963+0000","last_peered":"2024-04-27T07:05:02.903963+0000","last_clean":"2024-04-27T07:05:02.903963+0000","last_became_active":"2024-04-27T07:05:01.612680+0000","last_became_peered":"2024-04-27T07:05:01.612680+0000","last_unstale":"2024-04-27T07:05:02.903963+0000","last_undegraded":"2024-04-27T07:05:02.903963+0000","last_fullsized":"2024-04-27T07:05:02.903963+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:16:21.551875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902590+0000","last_change":"2024-04-27T07:05:01.619583+0000","last_active":"2024-04-27T07:05:02.902590+0000","last_peered":"2024-04-27T07:05:02.902590+0000","last_clean":"2024-04-27T07:05:02.902590+0000","last_became_active":"2024-04-27T07:05:01.619090+0000","last_became_peered":"2024-04-27T07:05:01.619090+0000","last_unstale":"2024-04-27T07:05:02.902590+0000","last_undegraded":"2024-04-27T07:05:02.902590+0000","last_fullsized":"2024-04-27T07:05:02.902590+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:57:46.218063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902222+0000","last_change":"2024-04-27T07:05:01.619624+0000","last_active":"2024-04-27T07:05:02.902222+0000","last_peered":"2024-04-27T07:05:02.902222+0000","last_clean":"2024-04-27T07:05:02.902222+0000","last_became_active":"2024-04-27T07:05:01.618807+0000","last_became_peered":"2024-04-27T07:05:01.618807+0000","last_unstale":"2024-04-27T07:05:02.902222+0000","last_undegraded":"2024-04-27T07:05:02.902222+0000","last_fullsized":"2024-04-27T07:05:02.902222+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:27:03.887951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577135+0000","last_change":"2024-04-27T07:05:01.593491+0000","last_active":"2024-04-27T07:05:02.577135+0000","last_peered":"2024-04-27T07:05:02.577135+0000","last_clean":"2024-04-27T07:05:02.577135+0000","last_became_active":"2024-04-27T07:05:01.592754+0000","last_became_peered":"2024-04-27T07:05:01.592754+0000","last_unstale":"2024-04-27T07:05:02.577135+0000","last_undegraded":"2024-04-27T07:05:02.577135+0000","last_fullsized":"2024-04-27T07:05:02.577135+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:07:11.751089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906136+0000","last_change":"2024-04-27T07:05:01.596304+0000","last_active":"2024-04-27T07:05:02.906136+0000","last_peered":"2024-04-27T07:05:02.906136+0000","last_clean":"2024-04-27T07:05:02.906136+0000","last_became_active":"2024-04-27T07:05:01.595898+0000","last_became_peered":"2024-04-27T07:05:01.595898+0000","last_unstale":"2024-04-27T07:05:02.906136+0000","last_undegraded":"2024-04-27T07:05:02.906136+0000","last_fullsized":"2024-04-27T07:05:02.906136+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:45:14.410767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902886+0000","last_change":"2024-04-27T07:05:01.615052+0000","last_active":"2024-04-27T07:05:02.902886+0000","last_peered":"2024-04-27T07:05:02.902886+0000","last_clean":"2024-04-27T07:05:02.902886+0000","last_became_active":"2024-04-27T07:05:01.614397+0000","last_became_peered":"2024-04-27T07:05:01.614397+0000","last_unstale":"2024-04-27T07:05:02.902886+0000","last_undegraded":"2024-04-27T07:05:02.902886+0000","last_fullsized":"2024-04-27T07:05:02.902886+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:57:13.964316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903764+0000","last_change":"2024-04-27T07:05:01.611400+0000","last_active":"2024-04-27T07:05:02.903764+0000","last_peered":"2024-04-27T07:05:02.903764+0000","last_clean":"2024-04-27T07:05:02.903764+0000","last_became_active":"2024-04-27T07:05:01.610436+0000","last_became_peered":"2024-04-27T07:05:01.610436+0000","last_unstale":"2024-04-27T07:05:02.903764+0000","last_undegraded":"2024-04-27T07:05:02.903764+0000","last_fullsized":"2024-04-27T07:05:02.903764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:11.729329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576774+0000","last_change":"2024-04-27T07:05:01.589627+0000","last_active":"2024-04-27T07:05:02.576774+0000","last_peered":"2024-04-27T07:05:02.576774+0000","last_clean":"2024-04-27T07:05:02.576774+0000","last_became_active":"2024-04-27T07:05:01.589376+0000","last_became_peered":"2024-04-27T07:05:01.589376+0000","last_unstale":"2024-04-27T07:05:02.576774+0000","last_undegraded":"2024-04-27T07:05:02.576774+0000","last_fullsized":"2024-04-27T07:05:02.576774+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:11:15.532969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904361+0000","last_change":"2024-04-27T07:05:01.597525+0000","last_active":"2024-04-27T07:05:02.904361+0000","last_peered":"2024-04-27T07:05:02.904361+0000","last_clean":"2024-04-27T07:05:02.904361+0000","last_became_active":"2024-04-27T07:05:01.596969+0000","last_became_peered":"2024-04-27T07:05:01.596969+0000","last_unstale":"2024-04-27T07:05:02.904361+0000","last_undegraded":"2024-04-27T07:05:02.904361+0000","last_fullsized":"2024-04-27T07:05:02.904361+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:57:13.964316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903342+0000","last_change":"2024-04-27T07:05:01.615038+0000","last_active":"2024-04-27T07:05:02.903342+0000","last_peered":"2024-04-27T07:05:02.903342+0000","last_clean":"2024-04-27T07:05:02.903342+0000","last_became_active":"2024-04-27T07:05:01.614661+0000","last_became_peered":"2024-04-27T07:05:01.614661+0000","last_unstale":"2024-04-27T07:05:02.903342+0000","last_undegraded":"2024-04-27T07:05:02.903342+0000","last_fullsized":"2024-04-27T07:05:02.903342+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:27:03.887951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903121+0000","last_change":"2024-04-27T07:05:01.625925+0000","last_active":"2024-04-27T07:05:02.903121+0000","last_peered":"2024-04-27T07:05:02.903121+0000","last_clean":"2024-04-27T07:05:02.903121+0000","last_became_active":"2024-04-27T07:05:01.625265+0000","last_became_peered":"2024-04-27T07:05:01.625265+0000","last_unstale":"2024-04-27T07:05:02.903121+0000","last_undegraded":"2024-04-27T07:05:02.903121+0000","last_fullsized":"2024-04-27T07:05:02.903121+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:43:55.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903799+0000","last_change":"2024-04-27T07:05:01.620564+0000","last_active":"2024-04-27T07:05:02.903799+0000","last_peered":"2024-04-27T07:05:02.903799+0000","last_clean":"2024-04-27T07:05:02.903799+0000","last_became_active":"2024-04-27T07:05:01.620060+0000","last_became_peered":"2024-04-27T07:05:01.620060+0000","last_unstale":"2024-04-27T07:05:02.903799+0000","last_undegraded":"2024-04-27T07:05:02.903799+0000","last_fullsized":"2024-04-27T07:05:02.903799+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:54:27.931459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904353+0000","last_change":"2024-04-27T07:05:01.595065+0000","last_active":"2024-04-27T07:05:02.904353+0000","last_peered":"2024-04-27T07:05:02.904353+0000","last_clean":"2024-04-27T07:05:02.904353+0000","last_became_active":"2024-04-27T07:05:01.594768+0000","last_became_peered":"2024-04-27T07:05:01.594768+0000","last_unstale":"2024-04-27T07:05:02.904353+0000","last_undegraded":"2024-04-27T07:05:02.904353+0000","last_fullsized":"2024-04-27T07:05:02.904353+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:22:36.544610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904007+0000","last_change":"2024-04-27T07:05:01.596597+0000","last_active":"2024-04-27T07:05:02.904007+0000","last_peered":"2024-04-27T07:05:02.904007+0000","last_clean":"2024-04-27T07:05:02.904007+0000","last_became_active":"2024-04-27T07:05:01.594969+0000","last_became_peered":"2024-04-27T07:05:01.594969+0000","last_unstale":"2024-04-27T07:05:02.904007+0000","last_undegraded":"2024-04-27T07:05:02.904007+0000","last_fullsized":"2024-04-27T07:05:02.904007+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:57:46.218063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576307+0000","last_change":"2024-04-27T07:05:01.612692+0000","last_active":"2024-04-27T07:05:02.576307+0000","last_peered":"2024-04-27T07:05:02.576307+0000","last_clean":"2024-04-27T07:05:02.576307+0000","last_became_active":"2024-04-27T07:05:01.612263+0000","last_became_peered":"2024-04-27T07:05:01.612263+0000","last_unstale":"2024-04-27T07:05:02.576307+0000","last_undegraded":"2024-04-27T07:05:02.576307+0000","last_fullsized":"2024-04-27T07:05:02.576307+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:07:53.287473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574899+0000","last_change":"2024-04-27T07:05:01.579406+0000","last_active":"2024-04-27T07:05:02.574899+0000","last_peered":"2024-04-27T07:05:02.574899+0000","last_clean":"2024-04-27T07:05:02.574899+0000","last_became_active":"2024-04-27T07:05:01.578630+0000","last_became_peered":"2024-04-27T07:05:01.578630+0000","last_unstale":"2024-04-27T07:05:02.574899+0000","last_undegraded":"2024-04-27T07:05:02.574899+0000","last_fullsized":"2024-04-27T07:05:02.574899+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:33:20.122035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574752+0000","last_change":"2024-04-27T07:05:01.601560+0000","last_active":"2024-04-27T07:05:02.574752+0000","last_peered":"2024-04-27T07:05:02.574752+0000","last_clean":"2024-04-27T07:05:02.574752+0000","last_became_active":"2024-04-27T07:05:01.600159+0000","last_became_peered":"2024-04-27T07:05:01.600159+0000","last_unstale":"2024-04-27T07:05:02.574752+0000","last_undegraded":"2024-04-27T07:05:02.574752+0000","last_fullsized":"2024-04-27T07:05:02.574752+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:41:35.033852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903224+0000","last_change":"2024-04-27T07:05:01.610220+0000","last_active":"2024-04-27T07:05:02.903224+0000","last_peered":"2024-04-27T07:05:02.903224+0000","last_clean":"2024-04-27T07:05:02.903224+0000","last_became_active":"2024-04-27T07:05:01.609992+0000","last_became_peered":"2024-04-27T07:05:01.609992+0000","last_unstale":"2024-04-27T07:05:02.903224+0000","last_undegraded":"2024-04-27T07:05:02.903224+0000","last_fullsized":"2024-04-27T07:05:02.903224+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:41:35.033852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903106+0000","last_change":"2024-04-27T07:05:01.597341+0000","last_active":"2024-04-27T07:05:02.903106+0000","last_peered":"2024-04-27T07:05:02.903106+0000","last_clean":"2024-04-27T07:05:02.903106+0000","last_became_active":"2024-04-27T07:05:01.596539+0000","last_became_peered":"2024-04-27T07:05:01.596539+0000","last_unstale":"2024-04-27T07:05:02.903106+0000","last_undegraded":"2024-04-27T07:05:02.903106+0000","last_fullsized":"2024-04-27T07:05:02.903106+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:50.115271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905837+0000","last_change":"2024-04-27T07:05:01.616680+0000","last_active":"2024-04-27T07:05:02.905837+0000","last_peered":"2024-04-27T07:05:02.905837+0000","last_clean":"2024-04-27T07:05:02.905837+0000","last_became_active":"2024-04-27T07:05:01.616383+0000","last_became_peered":"2024-04-27T07:05:01.616383+0000","last_unstale":"2024-04-27T07:05:02.905837+0000","last_undegraded":"2024-04-27T07:05:02.905837+0000","last_fullsized":"2024-04-27T07:05:02.905837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:22.396865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904732+0000","last_change":"2024-04-27T07:05:01.610200+0000","last_active":"2024-04-27T07:05:02.904732+0000","last_peered":"2024-04-27T07:05:02.904732+0000","last_clean":"2024-04-27T07:05:02.904732+0000","last_became_active":"2024-04-27T07:05:01.609866+0000","last_became_peered":"2024-04-27T07:05:01.609866+0000","last_unstale":"2024-04-27T07:05:02.904732+0000","last_undegraded":"2024-04-27T07:05:02.904732+0000","last_fullsized":"2024-04-27T07:05:02.904732+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:43:55.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.907760+0000","last_change":"2024-04-27T07:05:01.619276+0000","last_active":"2024-04-27T07:05:02.907760+0000","last_peered":"2024-04-27T07:05:02.907760+0000","last_clean":"2024-04-27T07:05:02.907760+0000","last_became_active":"2024-04-27T07:05:01.618963+0000","last_became_peered":"2024-04-27T07:05:01.618963+0000","last_unstale":"2024-04-27T07:05:02.907760+0000","last_undegraded":"2024-04-27T07:05:02.907760+0000","last_fullsized":"2024-04-27T07:05:02.907760+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:25:16.721450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904178+0000","last_change":"2024-04-27T07:05:01.596015+0000","last_active":"2024-04-27T07:05:02.904178+0000","last_peered":"2024-04-27T07:05:02.904178+0000","last_clean":"2024-04-27T07:05:02.904178+0000","last_became_active":"2024-04-27T07:05:01.595230+0000","last_became_peered":"2024-04-27T07:05:01.595230+0000","last_unstale":"2024-04-27T07:05:02.904178+0000","last_undegraded":"2024-04-27T07:05:02.904178+0000","last_fullsized":"2024-04-27T07:05:02.904178+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:25:52.050396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575170+0000","last_change":"2024-04-27T07:05:01.602051+0000","last_active":"2024-04-27T07:05:02.575170+0000","last_peered":"2024-04-27T07:05:02.575170+0000","last_clean":"2024-04-27T07:05:02.575170+0000","last_became_active":"2024-04-27T07:05:01.600826+0000","last_became_peered":"2024-04-27T07:05:01.600826+0000","last_unstale":"2024-04-27T07:05:02.575170+0000","last_undegraded":"2024-04-27T07:05:02.575170+0000","last_fullsized":"2024-04-27T07:05:02.575170+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:21.852593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904746+0000","last_change":"2024-04-27T07:05:01.619580+0000","last_active":"2024-04-27T07:05:02.904746+0000","last_peered":"2024-04-27T07:05:02.904746+0000","last_clean":"2024-04-27T07:05:02.904746+0000","last_became_active":"2024-04-27T07:05:01.618840+0000","last_became_peered":"2024-04-27T07:05:01.618840+0000","last_unstale":"2024-04-27T07:05:02.904746+0000","last_undegraded":"2024-04-27T07:05:02.904746+0000","last_fullsized":"2024-04-27T07:05:02.904746+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:54.205836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576911+0000","last_change":"2024-04-27T07:05:01.611749+0000","last_active":"2024-04-27T07:05:02.576911+0000","last_peered":"2024-04-27T07:05:02.576911+0000","last_clean":"2024-04-27T07:05:02.576911+0000","last_became_active":"2024-04-27T07:05:01.610895+0000","last_became_peered":"2024-04-27T07:05:01.610895+0000","last_unstale":"2024-04-27T07:05:02.576911+0000","last_undegraded":"2024-04-27T07:05:02.576911+0000","last_fullsized":"2024-04-27T07:05:02.576911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:44.123073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902226+0000","last_change":"2024-04-27T07:05:01.630879+0000","last_active":"2024-04-27T07:05:02.902226+0000","last_peered":"2024-04-27T07:05:02.902226+0000","last_clean":"2024-04-27T07:05:02.902226+0000","last_became_active":"2024-04-27T07:05:01.630279+0000","last_became_peered":"2024-04-27T07:05:01.630279+0000","last_unstale":"2024-04-27T07:05:02.902226+0000","last_undegraded":"2024-04-27T07:05:02.902226+0000","last_fullsized":"2024-04-27T07:05:02.902226+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:11.729329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906808+0000","last_change":"2024-04-27T07:05:01.610677+0000","last_active":"2024-04-27T07:05:02.906808+0000","last_peered":"2024-04-27T07:05:02.906808+0000","last_clean":"2024-04-27T07:05:02.906808+0000","last_became_active":"2024-04-27T07:05:01.610473+0000","last_became_peered":"2024-04-27T07:05:01.610473+0000","last_unstale":"2024-04-27T07:05:02.906808+0000","last_undegraded":"2024-04-27T07:05:02.906808+0000","last_fullsized":"2024-04-27T07:05:02.906808+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:24.705960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"28'4","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577957+0000","last_change":"2024-04-27T07:05:00.576697+0000","last_active":"2024-04-27T07:05:02.577957+0000","last_peered":"2024-04-27T07:05:02.577957+0000","last_clean":"2024-04-27T07:05:02.577957+0000","last_became_active":"2024-04-27T07:05:00.576468+0000","last_became_peered":"2024-04-27T07:05:00.576468+0000","last_unstale":"2024-04-27T07:05:02.577957+0000","last_undegraded":"2024-04-27T07:05:02.577957+0000","last_fullsized":"2024-04-27T07:05:02.577957+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:02.662003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577665+0000","last_change":"2024-04-27T07:05:01.582659+0000","last_active":"2024-04-27T07:05:02.577665+0000","last_peered":"2024-04-27T07:05:02.577665+0000","last_clean":"2024-04-27T07:05:02.577665+0000","last_became_active":"2024-04-27T07:05:01.581933+0000","last_became_peered":"2024-04-27T07:05:01.581933+0000","last_unstale":"2024-04-27T07:05:02.577665+0000","last_undegraded":"2024-04-27T07:05:02.577665+0000","last_fullsized":"2024-04-27T07:05:02.577665+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:41:35.033852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906735+0000","last_change":"2024-04-27T07:05:00.568249+0000","last_active":"2024-04-27T07:05:02.906735+0000","last_peered":"2024-04-27T07:05:02.906735+0000","last_clean":"2024-04-27T07:05:02.906735+0000","last_became_active":"2024-04-27T07:05:00.567955+0000","last_became_peered":"2024-04-27T07:05:00.567955+0000","last_unstale":"2024-04-27T07:05:02.906735+0000","last_undegraded":"2024-04-27T07:05:02.906735+0000","last_fullsized":"2024-04-27T07:05:02.906735+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:39.842835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575425+0000","last_change":"2024-04-27T07:05:01.603708+0000","last_active":"2024-04-27T07:05:02.575425+0000","last_peered":"2024-04-27T07:05:02.575425+0000","last_clean":"2024-04-27T07:05:02.575425+0000","last_became_active":"2024-04-27T07:05:01.602794+0000","last_became_peered":"2024-04-27T07:05:01.602794+0000","last_unstale":"2024-04-27T07:05:02.575425+0000","last_undegraded":"2024-04-27T07:05:02.575425+0000","last_fullsized":"2024-04-27T07:05:02.575425+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:24.705960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904301+0000","last_change":"2024-04-27T07:05:00.580459+0000","last_active":"2024-04-27T07:05:02.904301+0000","last_peered":"2024-04-27T07:05:02.904301+0000","last_clean":"2024-04-27T07:05:02.904301+0000","last_became_active":"2024-04-27T07:05:00.579931+0000","last_became_peered":"2024-04-27T07:05:00.579931+0000","last_unstale":"2024-04-27T07:05:02.904301+0000","last_undegraded":"2024-04-27T07:05:02.904301+0000","last_fullsized":"2024-04-27T07:05:02.904301+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:43.113828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.907636+0000","last_change":"2024-04-27T07:05:01.601059+0000","last_active":"2024-04-27T07:05:02.907636+0000","last_peered":"2024-04-27T07:05:02.907636+0000","last_clean":"2024-04-27T07:05:02.907636+0000","last_became_active":"2024-04-27T07:05:01.600683+0000","last_became_peered":"2024-04-27T07:05:01.600683+0000","last_unstale":"2024-04-27T07:05:02.907636+0000","last_undegraded":"2024-04-27T07:05:02.907636+0000","last_fullsized":"2024-04-27T07:05:02.907636+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:50:18.189750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576072+0000","last_change":"2024-04-27T07:05:00.564610+0000","last_active":"2024-04-27T07:05:02.576072+0000","last_peered":"2024-04-27T07:05:02.576072+0000","last_clean":"2024-04-27T07:05:02.576072+0000","last_became_active":"2024-04-27T07:05:00.564183+0000","last_became_peered":"2024-04-27T07:05:00.564183+0000","last_unstale":"2024-04-27T07:05:02.576072+0000","last_undegraded":"2024-04-27T07:05:02.576072+0000","last_fullsized":"2024-04-27T07:05:02.576072+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:39:49.203139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904960+0000","last_change":"2024-04-27T07:05:01.626297+0000","last_active":"2024-04-27T07:05:02.904960+0000","last_peered":"2024-04-27T07:05:02.904960+0000","last_clean":"2024-04-27T07:05:02.904960+0000","last_became_active":"2024-04-27T07:05:01.625643+0000","last_became_peered":"2024-04-27T07:05:01.625643+0000","last_unstale":"2024-04-27T07:05:02.904960+0000","last_undegraded":"2024-04-27T07:05:02.904960+0000","last_fullsized":"2024-04-27T07:05:02.904960+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:35.493213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575183+0000","last_change":"2024-04-27T07:05:00.580618+0000","last_active":"2024-04-27T07:05:02.575183+0000","last_peered":"2024-04-27T07:05:02.575183+0000","last_clean":"2024-04-27T07:05:02.575183+0000","last_became_active":"2024-04-27T07:05:00.579811+0000","last_became_peered":"2024-04-27T07:05:00.579811+0000","last_unstale":"2024-04-27T07:05:02.575183+0000","last_undegraded":"2024-04-27T07:05:02.575183+0000","last_fullsized":"2024-04-27T07:05:02.575183+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:03:30.967635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903759+0000","last_change":"2024-04-27T07:05:01.609943+0000","last_active":"2024-04-27T07:05:02.903759+0000","last_peered":"2024-04-27T07:05:02.903759+0000","last_clean":"2024-04-27T07:05:02.903759+0000","last_became_active":"2024-04-27T07:05:01.608942+0000","last_became_peered":"2024-04-27T07:05:01.608942+0000","last_unstale":"2024-04-27T07:05:02.903759+0000","last_undegraded":"2024-04-27T07:05:02.903759+0000","last_fullsized":"2024-04-27T07:05:02.903759+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:16:21.551875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574662+0000","last_change":"2024-04-27T07:05:00.583325+0000","last_active":"2024-04-27T07:05:02.574662+0000","last_peered":"2024-04-27T07:05:02.574662+0000","last_clean":"2024-04-27T07:05:02.574662+0000","last_became_active":"2024-04-27T07:05:00.581179+0000","last_became_peered":"2024-04-27T07:05:00.581179+0000","last_unstale":"2024-04-27T07:05:02.574662+0000","last_undegraded":"2024-04-27T07:05:02.574662+0000","last_fullsized":"2024-04-27T07:05:02.574662+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:04:27.277283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574396+0000","last_change":"2024-04-27T07:05:01.584737+0000","last_active":"2024-04-27T07:05:02.574396+0000","last_peered":"2024-04-27T07:05:02.574396+0000","last_clean":"2024-04-27T07:05:02.574396+0000","last_became_active":"2024-04-27T07:05:01.584001+0000","last_became_peered":"2024-04-27T07:05:01.584001+0000","last_unstale":"2024-04-27T07:05:02.574396+0000","last_undegraded":"2024-04-27T07:05:02.574396+0000","last_fullsized":"2024-04-27T07:05:02.574396+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:56:25.278857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902799+0000","last_change":"2024-04-27T07:05:00.576227+0000","last_active":"2024-04-27T07:05:02.902799+0000","last_peered":"2024-04-27T07:05:02.902799+0000","last_clean":"2024-04-27T07:05:02.902799+0000","last_became_active":"2024-04-27T07:05:00.575365+0000","last_became_peered":"2024-04-27T07:05:00.575365+0000","last_unstale":"2024-04-27T07:05:02.902799+0000","last_undegraded":"2024-04-27T07:05:02.902799+0000","last_fullsized":"2024-04-27T07:05:02.902799+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:24:20.674931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904849+0000","last_change":"2024-04-27T07:05:01.613711+0000","last_active":"2024-04-27T07:05:02.904849+0000","last_peered":"2024-04-27T07:05:02.904849+0000","last_clean":"2024-04-27T07:05:02.904849+0000","last_became_active":"2024-04-27T07:05:01.613373+0000","last_became_peered":"2024-04-27T07:05:01.613373+0000","last_unstale":"2024-04-27T07:05:02.904849+0000","last_undegraded":"2024-04-27T07:05:02.904849+0000","last_fullsized":"2024-04-27T07:05:02.904849+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:09:25.577049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574234+0000","last_change":"2024-04-27T07:05:00.566993+0000","last_active":"2024-04-27T07:05:02.574234+0000","last_peered":"2024-04-27T07:05:02.574234+0000","last_clean":"2024-04-27T07:05:02.574234+0000","last_became_active":"2024-04-27T07:05:00.566758+0000","last_became_peered":"2024-04-27T07:05:00.566758+0000","last_unstale":"2024-04-27T07:05:02.574234+0000","last_undegraded":"2024-04-27T07:05:02.574234+0000","last_fullsized":"2024-04-27T07:05:02.574234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:02.662003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905468+0000","last_change":"2024-04-27T07:05:01.597697+0000","last_active":"2024-04-27T07:05:02.905468+0000","last_peered":"2024-04-27T07:05:02.905468+0000","last_clean":"2024-04-27T07:05:02.905468+0000","last_became_active":"2024-04-27T07:05:01.596910+0000","last_became_peered":"2024-04-27T07:05:01.596910+0000","last_unstale":"2024-04-27T07:05:02.905468+0000","last_undegraded":"2024-04-27T07:05:02.905468+0000","last_fullsized":"2024-04-27T07:05:02.905468+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:57:13.964316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906248+0000","last_change":"2024-04-27T07:05:00.584576+0000","last_active":"2024-04-27T07:05:02.906248+0000","last_peered":"2024-04-27T07:05:02.906248+0000","last_clean":"2024-04-27T07:05:02.906248+0000","last_became_active":"2024-04-27T07:05:00.584108+0000","last_became_peered":"2024-04-27T07:05:00.584108+0000","last_unstale":"2024-04-27T07:05:02.906248+0000","last_undegraded":"2024-04-27T07:05:02.906248+0000","last_fullsized":"2024-04-27T07:05:02.906248+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:41:34.024607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906001+0000","last_change":"2024-04-27T07:05:01.595435+0000","last_active":"2024-04-27T07:05:02.906001+0000","last_peered":"2024-04-27T07:05:02.906001+0000","last_clean":"2024-04-27T07:05:02.906001+0000","last_became_active":"2024-04-27T07:05:01.594792+0000","last_became_peered":"2024-04-27T07:05:01.594792+0000","last_unstale":"2024-04-27T07:05:02.906001+0000","last_undegraded":"2024-04-27T07:05:02.906001+0000","last_fullsized":"2024-04-27T07:05:02.906001+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:54.205836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906884+0000","last_change":"2024-04-27T07:05:00.573881+0000","last_active":"2024-04-27T07:05:02.906884+0000","last_peered":"2024-04-27T07:05:02.906884+0000","last_clean":"2024-04-27T07:05:02.906884+0000","last_became_active":"2024-04-27T07:05:00.573338+0000","last_became_peered":"2024-04-27T07:05:00.573338+0000","last_unstale":"2024-04-27T07:05:02.906884+0000","last_undegraded":"2024-04-27T07:05:02.906884+0000","last_fullsized":"2024-04-27T07:05:02.906884+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:22.266250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904246+0000","last_change":"2024-04-27T07:05:01.610120+0000","last_active":"2024-04-27T07:05:02.904246+0000","last_peered":"2024-04-27T07:05:02.904246+0000","last_clean":"2024-04-27T07:05:02.904246+0000","last_became_active":"2024-04-27T07:05:01.609805+0000","last_became_peered":"2024-04-27T07:05:01.609805+0000","last_unstale":"2024-04-27T07:05:02.904246+0000","last_undegraded":"2024-04-27T07:05:02.904246+0000","last_fullsized":"2024-04-27T07:05:02.904246+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:25:52.050396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"28'3","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904476+0000","last_change":"2024-04-27T07:05:00.574817+0000","last_active":"2024-04-27T07:05:02.904476+0000","last_peered":"2024-04-27T07:05:02.904476+0000","last_clean":"2024-04-27T07:05:02.904476+0000","last_became_active":"2024-04-27T07:05:00.573530+0000","last_became_peered":"2024-04-27T07:05:00.573530+0000","last_unstale":"2024-04-27T07:05:02.904476+0000","last_undegraded":"2024-04-27T07:05:02.904476+0000","last_fullsized":"2024-04-27T07:05:02.904476+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:33:19.112790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904123+0000","last_change":"2024-04-27T07:05:01.609943+0000","last_active":"2024-04-27T07:05:02.904123+0000","last_peered":"2024-04-27T07:05:02.904123+0000","last_clean":"2024-04-27T07:05:02.904123+0000","last_became_active":"2024-04-27T07:05:01.609179+0000","last_became_peered":"2024-04-27T07:05:01.609179+0000","last_unstale":"2024-04-27T07:05:02.904123+0000","last_undegraded":"2024-04-27T07:05:02.904123+0000","last_fullsized":"2024-04-27T07:05:02.904123+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:57:46.218063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"28'6","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904519+0000","last_change":"2024-04-27T07:05:00.581950+0000","last_active":"2024-04-27T07:05:02.904519+0000","last_peered":"2024-04-27T07:05:02.904519+0000","last_clean":"2024-04-27T07:05:02.904519+0000","last_became_active":"2024-04-27T07:05:00.581540+0000","last_became_peered":"2024-04-27T07:05:00.581540+0000","last_unstale":"2024-04-27T07:05:02.904519+0000","last_undegraded":"2024-04-27T07:05:02.904519+0000","last_fullsized":"2024-04-27T07:05:02.904519+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:33:19.112790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904841+0000","last_change":"2024-04-27T07:05:01.613661+0000","last_active":"2024-04-27T07:05:02.904841+0000","last_peered":"2024-04-27T07:05:02.904841+0000","last_clean":"2024-04-27T07:05:02.904841+0000","last_became_active":"2024-04-27T07:05:01.613346+0000","last_became_peered":"2024-04-27T07:05:01.613346+0000","last_unstale":"2024-04-27T07:05:02.904841+0000","last_undegraded":"2024-04-27T07:05:02.904841+0000","last_fullsized":"2024-04-27T07:05:02.904841+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:24.705960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575383+0000","last_change":"2024-04-27T07:05:00.563391+0000","last_active":"2024-04-27T07:05:02.575383+0000","last_peered":"2024-04-27T07:05:02.575383+0000","last_clean":"2024-04-27T07:05:02.575383+0000","last_became_active":"2024-04-27T07:05:00.563127+0000","last_became_peered":"2024-04-27T07:05:00.563127+0000","last_unstale":"2024-04-27T07:05:02.575383+0000","last_undegraded":"2024-04-27T07:05:02.575383+0000","last_fullsized":"2024-04-27T07:05:02.575383+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:53:39.852098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905410+0000","last_change":"2024-04-27T07:05:01.583757+0000","last_active":"2024-04-27T07:05:02.905410+0000","last_peered":"2024-04-27T07:05:02.905410+0000","last_clean":"2024-04-27T07:05:02.905410+0000","last_became_active":"2024-04-27T07:05:01.583487+0000","last_became_peered":"2024-04-27T07:05:01.583487+0000","last_unstale":"2024-04-27T07:05:02.905410+0000","last_undegraded":"2024-04-27T07:05:02.905410+0000","last_fullsized":"2024-04-27T07:05:02.905410+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:54:27.931459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905265+0000","last_change":"2024-04-27T07:05:00.578652+0000","last_active":"2024-04-27T07:05:02.905265+0000","last_peered":"2024-04-27T07:05:02.905265+0000","last_clean":"2024-04-27T07:05:02.905265+0000","last_became_active":"2024-04-27T07:05:00.577937+0000","last_became_peered":"2024-04-27T07:05:00.577937+0000","last_unstale":"2024-04-27T07:05:02.905265+0000","last_undegraded":"2024-04-27T07:05:02.905265+0000","last_fullsized":"2024-04-27T07:05:02.905265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:39.842835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902661+0000","last_change":"2024-04-27T07:05:01.626608+0000","last_active":"2024-04-27T07:05:02.902661+0000","last_peered":"2024-04-27T07:05:02.902661+0000","last_clean":"2024-04-27T07:05:02.902661+0000","last_became_active":"2024-04-27T07:05:01.626030+0000","last_became_peered":"2024-04-27T07:05:01.626030+0000","last_unstale":"2024-04-27T07:05:02.902661+0000","last_undegraded":"2024-04-27T07:05:02.902661+0000","last_fullsized":"2024-04-27T07:05:02.902661+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:25:52.050396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903051+0000","last_change":"2024-04-27T07:05:00.582663+0000","last_active":"2024-04-27T07:05:02.903051+0000","last_peered":"2024-04-27T07:05:02.903051+0000","last_clean":"2024-04-27T07:05:02.903051+0000","last_became_active":"2024-04-27T07:05:00.582130+0000","last_became_peered":"2024-04-27T07:05:00.582130+0000","last_unstale":"2024-04-27T07:05:02.903051+0000","last_undegraded":"2024-04-27T07:05:02.903051+0000","last_fullsized":"2024-04-27T07:05:02.903051+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:35:03.637726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577385+0000","last_change":"2024-04-27T07:05:01.611934+0000","last_active":"2024-04-27T07:05:02.577385+0000","last_peered":"2024-04-27T07:05:02.577385+0000","last_clean":"2024-04-27T07:05:02.577385+0000","last_became_active":"2024-04-27T07:05:01.611104+0000","last_became_peered":"2024-04-27T07:05:01.611104+0000","last_unstale":"2024-04-27T07:05:02.577385+0000","last_undegraded":"2024-04-27T07:05:02.577385+0000","last_fullsized":"2024-04-27T07:05:02.577385+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:04:17.124168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"28'8","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903216+0000","last_change":"2024-04-27T07:05:00.582401+0000","last_active":"2024-04-27T07:05:02.903216+0000","last_peered":"2024-04-27T07:05:02.903216+0000","last_clean":"2024-04-27T07:05:02.903216+0000","last_became_active":"2024-04-27T07:05:00.582059+0000","last_became_peered":"2024-04-27T07:05:00.582059+0000","last_unstale":"2024-04-27T07:05:02.903216+0000","last_undegraded":"2024-04-27T07:05:02.903216+0000","last_fullsized":"2024-04-27T07:05:02.903216+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:56:24.269613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576437+0000","last_change":"2024-04-27T07:05:01.624915+0000","last_active":"2024-04-27T07:05:02.576437+0000","last_peered":"2024-04-27T07:05:02.576437+0000","last_clean":"2024-04-27T07:05:02.576437+0000","last_became_active":"2024-04-27T07:05:01.624142+0000","last_became_peered":"2024-04-27T07:05:01.624142+0000","last_unstale":"2024-04-27T07:05:02.576437+0000","last_undegraded":"2024-04-27T07:05:02.576437+0000","last_fullsized":"2024-04-27T07:05:02.576437+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:09:25.577049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906001+0000","last_change":"2024-04-27T07:05:00.580754+0000","last_active":"2024-04-27T07:05:02.906001+0000","last_peered":"2024-04-27T07:05:02.906001+0000","last_clean":"2024-04-27T07:05:02.906001+0000","last_became_active":"2024-04-27T07:05:00.579170+0000","last_became_peered":"2024-04-27T07:05:00.579170+0000","last_unstale":"2024-04-27T07:05:02.906001+0000","last_undegraded":"2024-04-27T07:05:02.906001+0000","last_fullsized":"2024-04-27T07:05:02.906001+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:36:50.917879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905657+0000","last_change":"2024-04-27T07:05:01.598762+0000","last_active":"2024-04-27T07:05:02.905657+0000","last_peered":"2024-04-27T07:05:02.905657+0000","last_clean":"2024-04-27T07:05:02.905657+0000","last_became_active":"2024-04-27T07:05:01.596478+0000","last_became_peered":"2024-04-27T07:05:01.596478+0000","last_unstale":"2024-04-27T07:05:02.905657+0000","last_undegraded":"2024-04-27T07:05:02.905657+0000","last_fullsized":"2024-04-27T07:05:02.905657+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:54:27.931459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576485+0000","last_change":"2024-04-27T07:05:00.581490+0000","last_active":"2024-04-27T07:05:02.576485+0000","last_peered":"2024-04-27T07:05:02.576485+0000","last_clean":"2024-04-27T07:05:02.576485+0000","last_became_active":"2024-04-27T07:05:00.581200+0000","last_became_peered":"2024-04-27T07:05:00.581200+0000","last_unstale":"2024-04-27T07:05:02.576485+0000","last_undegraded":"2024-04-27T07:05:02.576485+0000","last_fullsized":"2024-04-27T07:05:02.576485+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:11:14.523724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905429+0000","last_change":"2024-04-27T07:05:01.600930+0000","last_active":"2024-04-27T07:05:02.905429+0000","last_peered":"2024-04-27T07:05:02.905429+0000","last_clean":"2024-04-27T07:05:02.905429+0000","last_became_active":"2024-04-27T07:05:01.600645+0000","last_became_peered":"2024-04-27T07:05:01.600645+0000","last_unstale":"2024-04-27T07:05:02.905429+0000","last_undegraded":"2024-04-27T07:05:02.905429+0000","last_fullsized":"2024-04-27T07:05:02.905429+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:53:54.641089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576295+0000","last_change":"2024-04-27T07:05:00.563606+0000","last_active":"2024-04-27T07:05:02.576295+0000","last_peered":"2024-04-27T07:05:02.576295+0000","last_clean":"2024-04-27T07:05:02.576295+0000","last_became_active":"2024-04-27T07:05:00.563066+0000","last_became_peered":"2024-04-27T07:05:00.563066+0000","last_unstale":"2024-04-27T07:05:02.576295+0000","last_undegraded":"2024-04-27T07:05:02.576295+0000","last_fullsized":"2024-04-27T07:05:02.576295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:46.697491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576602+0000","last_change":"2024-04-27T07:05:01.609588+0000","last_active":"2024-04-27T07:05:02.576602+0000","last_peered":"2024-04-27T07:05:02.576602+0000","last_clean":"2024-04-27T07:05:02.576602+0000","last_became_active":"2024-04-27T07:05:01.609314+0000","last_became_peered":"2024-04-27T07:05:01.609314+0000","last_unstale":"2024-04-27T07:05:02.576602+0000","last_undegraded":"2024-04-27T07:05:02.576602+0000","last_fullsized":"2024-04-27T07:05:02.576602+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:53:40.861342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575570+0000","last_change":"2024-04-27T07:05:00.579714+0000","last_active":"2024-04-27T07:05:02.575570+0000","last_peered":"2024-04-27T07:05:02.575570+0000","last_clean":"2024-04-27T07:05:02.575570+0000","last_became_active":"2024-04-27T07:05:00.579470+0000","last_became_peered":"2024-04-27T07:05:00.579470+0000","last_unstale":"2024-04-27T07:05:02.575570+0000","last_undegraded":"2024-04-27T07:05:02.575570+0000","last_fullsized":"2024-04-27T07:05:02.575570+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:07:52.278228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576060+0000","last_change":"2024-04-27T07:05:01.589744+0000","last_active":"2024-04-27T07:05:02.576060+0000","last_peered":"2024-04-27T07:05:02.576060+0000","last_clean":"2024-04-27T07:05:02.576060+0000","last_became_active":"2024-04-27T07:05:01.586706+0000","last_became_peered":"2024-04-27T07:05:01.586706+0000","last_unstale":"2024-04-27T07:05:02.576060+0000","last_undegraded":"2024-04-27T07:05:02.576060+0000","last_fullsized":"2024-04-27T07:05:02.576060+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:47.706736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904096+0000","last_change":"2024-04-27T07:05:00.564682+0000","last_active":"2024-04-27T07:05:02.904096+0000","last_peered":"2024-04-27T07:05:02.904096+0000","last_clean":"2024-04-27T07:05:02.904096+0000","last_became_active":"2024-04-27T07:05:00.564229+0000","last_became_peered":"2024-04-27T07:05:00.564229+0000","last_unstale":"2024-04-27T07:05:02.904096+0000","last_undegraded":"2024-04-27T07:05:02.904096+0000","last_fullsized":"2024-04-27T07:05:02.904096+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:36:50.917879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576965+0000","last_change":"2024-04-27T07:05:01.587946+0000","last_active":"2024-04-27T07:05:02.576965+0000","last_peered":"2024-04-27T07:05:02.576965+0000","last_clean":"2024-04-27T07:05:02.576965+0000","last_became_active":"2024-04-27T07:05:01.587629+0000","last_became_peered":"2024-04-27T07:05:01.587629+0000","last_unstale":"2024-04-27T07:05:02.576965+0000","last_undegraded":"2024-04-27T07:05:02.576965+0000","last_fullsized":"2024-04-27T07:05:02.576965+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:04:28.286527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577895+0000","last_change":"2024-04-27T07:05:00.565169+0000","last_active":"2024-04-27T07:05:02.577895+0000","last_peered":"2024-04-27T07:05:02.577895+0000","last_clean":"2024-04-27T07:05:02.577895+0000","last_became_active":"2024-04-27T07:05:00.564908+0000","last_became_peered":"2024-04-27T07:05:00.564908+0000","last_unstale":"2024-04-27T07:05:02.577895+0000","last_undegraded":"2024-04-27T07:05:02.577895+0000","last_fullsized":"2024-04-27T07:05:02.577895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:32:39.772551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575595+0000","last_change":"2024-04-27T07:05:01.578774+0000","last_active":"2024-04-27T07:05:02.575595+0000","last_peered":"2024-04-27T07:05:02.575595+0000","last_clean":"2024-04-27T07:05:02.575595+0000","last_became_active":"2024-04-27T07:05:01.578476+0000","last_became_peered":"2024-04-27T07:05:01.578476+0000","last_unstale":"2024-04-27T07:05:02.575595+0000","last_undegraded":"2024-04-27T07:05:02.575595+0000","last_fullsized":"2024-04-27T07:05:02.575595+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:32.465776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574722+0000","last_change":"2024-04-27T07:05:00.571048+0000","last_active":"2024-04-27T07:05:02.574722+0000","last_peered":"2024-04-27T07:05:02.574722+0000","last_clean":"2024-04-27T07:05:02.574722+0000","last_became_active":"2024-04-27T07:05:00.570335+0000","last_became_peered":"2024-04-27T07:05:00.570335+0000","last_unstale":"2024-04-27T07:05:02.574722+0000","last_undegraded":"2024-04-27T07:05:02.574722+0000","last_fullsized":"2024-04-27T07:05:02.574722+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:31.456531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902265+0000","last_change":"2024-04-27T07:05:01.622320+0000","last_active":"2024-04-27T07:05:02.902265+0000","last_peered":"2024-04-27T07:05:02.902265+0000","last_clean":"2024-04-27T07:05:02.902265+0000","last_became_active":"2024-04-27T07:05:01.621631+0000","last_became_peered":"2024-04-27T07:05:01.621631+0000","last_unstale":"2024-04-27T07:05:02.902265+0000","last_undegraded":"2024-04-27T07:05:02.902265+0000","last_fullsized":"2024-04-27T07:05:02.902265+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:16:21.551875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903256+0000","last_change":"2024-04-27T07:05:00.576049+0000","last_active":"2024-04-27T07:05:02.903256+0000","last_peered":"2024-04-27T07:05:02.903256+0000","last_clean":"2024-04-27T07:05:02.903256+0000","last_became_active":"2024-04-27T07:05:00.575138+0000","last_became_peered":"2024-04-27T07:05:00.575138+0000","last_unstale":"2024-04-27T07:05:02.903256+0000","last_undegraded":"2024-04-27T07:05:02.903256+0000","last_fullsized":"2024-04-27T07:05:02.903256+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:41:34.024607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576480+0000","last_change":"2024-04-27T07:05:01.615732+0000","last_active":"2024-04-27T07:05:02.576480+0000","last_peered":"2024-04-27T07:05:02.576480+0000","last_clean":"2024-04-27T07:05:02.576480+0000","last_became_active":"2024-04-27T07:05:01.615515+0000","last_became_peered":"2024-04-27T07:05:01.615515+0000","last_unstale":"2024-04-27T07:05:02.576480+0000","last_undegraded":"2024-04-27T07:05:02.576480+0000","last_fullsized":"2024-04-27T07:05:02.576480+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:03:31.976880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906190+0000","last_change":"2024-04-27T07:05:00.579267+0000","last_active":"2024-04-27T07:05:02.906190+0000","last_peered":"2024-04-27T07:05:02.906190+0000","last_clean":"2024-04-27T07:05:02.906190+0000","last_became_active":"2024-04-27T07:05:00.578735+0000","last_became_peered":"2024-04-27T07:05:00.578735+0000","last_unstale":"2024-04-27T07:05:02.906190+0000","last_undegraded":"2024-04-27T07:05:02.906190+0000","last_fullsized":"2024-04-27T07:05:02.906190+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:32:39.772551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905648+0000","last_change":"2024-04-27T07:05:01.603236+0000","last_active":"2024-04-27T07:05:02.905648+0000","last_peered":"2024-04-27T07:05:02.905648+0000","last_clean":"2024-04-27T07:05:02.905648+0000","last_became_active":"2024-04-27T07:05:01.598615+0000","last_became_peered":"2024-04-27T07:05:01.598615+0000","last_unstale":"2024-04-27T07:05:02.905648+0000","last_undegraded":"2024-04-27T07:05:02.905648+0000","last_fullsized":"2024-04-27T07:05:02.905648+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:43:55.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.907094+0000","last_change":"2024-04-27T07:05:00.567386+0000","last_active":"2024-04-27T07:05:02.907094+0000","last_peered":"2024-04-27T07:05:02.907094+0000","last_clean":"2024-04-27T07:05:02.907094+0000","last_became_active":"2024-04-27T07:05:00.566965+0000","last_became_peered":"2024-04-27T07:05:00.566965+0000","last_unstale":"2024-04-27T07:05:02.907094+0000","last_undegraded":"2024-04-27T07:05:02.907094+0000","last_fullsized":"2024-04-27T07:05:02.907094+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:49.452830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905144+0000","last_change":"2024-04-27T07:05:01.582389+0000","last_active":"2024-04-27T07:05:02.905144+0000","last_peered":"2024-04-27T07:05:02.905144+0000","last_clean":"2024-04-27T07:05:02.905144+0000","last_became_active":"2024-04-27T07:05:01.582015+0000","last_became_peered":"2024-04-27T07:05:01.582015+0000","last_unstale":"2024-04-27T07:05:02.905144+0000","last_undegraded":"2024-04-27T07:05:02.905144+0000","last_fullsized":"2024-04-27T07:05:02.905144+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:21.852593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905687+0000","last_change":"2024-04-27T07:05:00.569614+0000","last_active":"2024-04-27T07:05:02.905687+0000","last_peered":"2024-04-27T07:05:02.905687+0000","last_clean":"2024-04-27T07:05:02.905687+0000","last_became_active":"2024-04-27T07:05:00.566282+0000","last_became_peered":"2024-04-27T07:05:00.566282+0000","last_unstale":"2024-04-27T07:05:02.905687+0000","last_undegraded":"2024-04-27T07:05:02.905687+0000","last_fullsized":"2024-04-27T07:05:02.905687+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:49.452830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.578103+0000","last_change":"2024-04-27T07:05:01.587208+0000","last_active":"2024-04-27T07:05:02.578103+0000","last_peered":"2024-04-27T07:05:02.578103+0000","last_clean":"2024-04-27T07:05:02.578103+0000","last_became_active":"2024-04-27T07:05:01.586403+0000","last_became_peered":"2024-04-27T07:05:01.586403+0000","last_unstale":"2024-04-27T07:05:02.578103+0000","last_undegraded":"2024-04-27T07:05:02.578103+0000","last_fullsized":"2024-04-27T07:05:02.578103+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:33:20.122035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904374+0000","last_change":"2024-04-27T07:05:00.576644+0000","last_active":"2024-04-27T07:05:02.904374+0000","last_peered":"2024-04-27T07:05:02.904374+0000","last_clean":"2024-04-27T07:05:02.904374+0000","last_became_active":"2024-04-27T07:05:00.576095+0000","last_became_peered":"2024-04-27T07:05:00.576095+0000","last_unstale":"2024-04-27T07:05:02.904374+0000","last_undegraded":"2024-04-27T07:05:02.904374+0000","last_fullsized":"2024-04-27T07:05:02.904374+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:32:39.772551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903816+0000","last_change":"2024-04-27T07:05:01.613416+0000","last_active":"2024-04-27T07:05:02.903816+0000","last_peered":"2024-04-27T07:05:02.903816+0000","last_clean":"2024-04-27T07:05:02.903816+0000","last_became_active":"2024-04-27T07:05:01.613055+0000","last_became_peered":"2024-04-27T07:05:01.613055+0000","last_unstale":"2024-04-27T07:05:02.903816+0000","last_undegraded":"2024-04-27T07:05:02.903816+0000","last_fullsized":"2024-04-27T07:05:02.903816+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:33:20.122035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905350+0000","last_change":"2024-04-27T07:05:00.577861+0000","last_active":"2024-04-27T07:05:02.905350+0000","last_peered":"2024-04-27T07:05:02.905350+0000","last_clean":"2024-04-27T07:05:02.905350+0000","last_became_active":"2024-04-27T07:05:00.577459+0000","last_became_peered":"2024-04-27T07:05:00.577459+0000","last_unstale":"2024-04-27T07:05:02.905350+0000","last_undegraded":"2024-04-27T07:05:02.905350+0000","last_fullsized":"2024-04-27T07:05:02.905350+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:22.266250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902897+0000","last_change":"2024-04-27T07:05:01.595839+0000","last_active":"2024-04-27T07:05:02.902897+0000","last_peered":"2024-04-27T07:05:02.902897+0000","last_clean":"2024-04-27T07:05:02.902897+0000","last_became_active":"2024-04-27T07:05:01.595069+0000","last_became_peered":"2024-04-27T07:05:01.595069+0000","last_unstale":"2024-04-27T07:05:02.902897+0000","last_undegraded":"2024-04-27T07:05:02.902897+0000","last_fullsized":"2024-04-27T07:05:02.902897+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:27:03.887951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574980+0000","last_change":"2024-04-27T07:05:00.579311+0000","last_active":"2024-04-27T07:05:02.574980+0000","last_peered":"2024-04-27T07:05:02.574980+0000","last_clean":"2024-04-27T07:05:02.574980+0000","last_became_active":"2024-04-27T07:05:00.578034+0000","last_became_peered":"2024-04-27T07:05:00.578034+0000","last_unstale":"2024-04-27T07:05:02.574980+0000","last_undegraded":"2024-04-27T07:05:02.574980+0000","last_fullsized":"2024-04-27T07:05:02.574980+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:35:41.861143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574462+0000","last_change":"2024-04-27T07:05:01.591150+0000","last_active":"2024-04-27T07:05:02.574462+0000","last_peered":"2024-04-27T07:05:02.574462+0000","last_clean":"2024-04-27T07:05:02.574462+0000","last_became_active":"2024-04-27T07:05:01.589801+0000","last_became_peered":"2024-04-27T07:05:01.589801+0000","last_unstale":"2024-04-27T07:05:02.574462+0000","last_undegraded":"2024-04-27T07:05:02.574462+0000","last_fullsized":"2024-04-27T07:05:02.574462+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:04:17.124168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903517+0000","last_change":"2024-04-27T07:05:00.584515+0000","last_active":"2024-04-27T07:05:02.903517+0000","last_peered":"2024-04-27T07:05:02.903517+0000","last_clean":"2024-04-27T07:05:02.903517+0000","last_became_active":"2024-04-27T07:05:00.583897+0000","last_became_peered":"2024-04-27T07:05:00.583897+0000","last_unstale":"2024-04-27T07:05:02.903517+0000","last_undegraded":"2024-04-27T07:05:02.903517+0000","last_fullsized":"2024-04-27T07:05:02.903517+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:04:16.114923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902658+0000","last_change":"2024-04-27T07:05:01.617423+0000","last_active":"2024-04-27T07:05:02.902658+0000","last_peered":"2024-04-27T07:05:02.902658+0000","last_clean":"2024-04-27T07:05:02.902658+0000","last_became_active":"2024-04-27T07:05:01.611893+0000","last_became_peered":"2024-04-27T07:05:01.611893+0000","last_unstale":"2024-04-27T07:05:02.902658+0000","last_undegraded":"2024-04-27T07:05:02.902658+0000","last_fullsized":"2024-04-27T07:05:02.902658+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:22:36.544610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576573+0000","last_change":"2024-04-27T07:05:00.572039+0000","last_active":"2024-04-27T07:05:02.576573+0000","last_peered":"2024-04-27T07:05:02.576573+0000","last_clean":"2024-04-27T07:05:02.576573+0000","last_became_active":"2024-04-27T07:05:00.571307+0000","last_became_peered":"2024-04-27T07:05:00.571307+0000","last_unstale":"2024-04-27T07:05:02.576573+0000","last_undegraded":"2024-04-27T07:05:02.576573+0000","last_fullsized":"2024-04-27T07:05:02.576573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:36:50.917879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903264+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:05:02.903264+0000","last_peered":"2024-04-27T07:05:02.903264+0000","last_clean":"2024-04-27T07:05:02.903264+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:05:02.903264+0000","last_undegraded":"2024-04-27T07:05:02.903264+0000","last_fullsized":"2024-04-27T07:05:02.903264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":714470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":331776,"data_stored":124628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":331776,"data_stored":124684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":331776,"data_stored":124116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":327680,"data_stored":119346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":124082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":125646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":714380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:05:05.656 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:05:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530bc1e640 1 Processor -- start 2024-04-27T07:05:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530bc1e640 1 -- start start 2024-04-27T07:05:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530bc1e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5304073c60 0x7f530409d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530bc1e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53040740a0 con 0x7f5304144fa0 2024-04-27T07:05:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530bc1e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53040b9df0 con 0x7f53040b9a10 2024-04-27T07:05:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530bc1e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f530409dec0 con 0x7f5304073c60 2024-04-27T07:05:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530a1bb640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5304144fa0 0x7f5304145430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35918/0 (socket says 172.21.15.12:35918) 2024-04-27T07:05:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530a1bb640 1 -- 172.21.15.12:0/3264847237 learned_addr learned my addr 172.21.15.12:0/3264847237 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53091b9640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5304073c60 0x7f530409d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3282052115 0 0) 0x7f53040740a0 con 0x7f5304144fa0 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52f0003530 con 0x7f5304144fa0 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2390739105 0 0) 0x7f52f0003530 con 0x7f5304144fa0 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5304073c60 msgr2=0x7f530409d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5304073c60 0x7f530409d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 >> v1:172.21.15.53:6789/0 conn(0x7f53040b9a10 legacy=0x7f530413e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530409e0e0 con 0x7f5304144fa0 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f52fc001a00 con 0x7f5304144fa0 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 >> v1:172.21.15.12:6789/0 conn(0x7f5304144fa0 legacy=0x7f5304145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5304073c60 0x7f52f00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52f0003f30 0x7f52f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52f0004d50 0x7f52f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f530409dec0 con 0x7f52f0004d50 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52f000ba30 con 0x7f52f0003f30 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f52f000bc30 con 0x7f5304073c60 2024-04-27T07:05:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530a1bb640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52f0004d50 0x7f52f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53091b9640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52f0003f30 0x7f52f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53099ba640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5304073c60 0x7f52f00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530a1bb640 1 -- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5304073c60 msgr2=0x7f52f00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:05.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530a1bb640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5304073c60 0x7f52f00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530a1bb640 1 -- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52f0003f30 msgr2=0x7f52f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:05.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530a1bb640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52f0003f30 0x7f52f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f530a1bb640 1 -- 172.21.15.12:0/3264847237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f0003530 con 0x7f52f0004d50 2024-04-27T07:05:05.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.812+0000 7f53091b9640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f52f0003f30 0x7f52f0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f530a1bb640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52f0004d50 0x7f52f00051d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00a850 tx=0x7f52fc01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308a3820c2add1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00a4a0 con 0x7f52f0004d50 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52fc0ad650 con 0x7f52f0004d50 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f53089b8640 1 -- 172.21.15.12:0/3264847237 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0ada80 con 0x7f52f0004d50 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f530bc1e640 1 -- 172.21.15.12:0/3264847237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52f0004d50 msgr2=0x7f52f00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f530bc1e640 1 --2- 172.21.15.12:0/3264847237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52f0004d50 0x7f52f00051d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00a850 tx=0x7f52fc01d360 comp rx=0 tx=0).stop 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f530a1bb640 1 -- 172.21.15.12:0/3264847237 reap_dead start 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f530bc1e640 1 -- 172.21.15.12:0/3264847237 shutdown_connections 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f530bc1e640 1 -- 172.21.15.12:0/3264847237 >> 172.21.15.12:0/3264847237 conn(0x7f5304067d10 msgr2=0x7f5304068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f530bc1e640 1 -- 172.21.15.12:0/3264847237 shutdown_connections 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f530bc1e640 1 -- 172.21.15.12:0/3264847237 wait complete. 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f530bc1e640 1 Processor -- start 2024-04-27T07:05:05.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.816+0000 7f530bc1e640 1 -- start start 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530bc1e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304144070 0x7f53041dab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530bc1e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f53040afc20 0x7f53040ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530bc1e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5304139560 0x7f53041399e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530bc1e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f530413eff0 con 0x7f5304139560 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530bc1e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5304144b10 con 0x7f5304144070 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530bc1e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f530413f170 con 0x7f53040afc20 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530a1bb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5304139560 0x7f53041399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530a1bb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5304139560 0x7f53041399e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58428/0 (socket says 172.21.15.12:58428) 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530a1bb640 1 -- 172.21.15.12:0/2928491545 learned_addr learned my addr 172.21.15.12:0/2928491545 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f53091b9640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f53040afc20 0x7f53040ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f53099ba640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304144070 0x7f53041dab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530a1bb640 1 -- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f53040afc20 msgr2=0x7f53040ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530a1bb640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f53040afc20 0x7f53040ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530a1bb640 1 -- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304144070 msgr2=0x7f53041dab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:05.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530a1bb640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304144070 0x7f53041dab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530a1bb640 1 -- 172.21.15.12:0/2928491545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530413c7a0 con 0x7f5304139560 2024-04-27T07:05:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530a1bb640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5304139560 0x7f53041399e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f52fc01cdd0 tx=0x7f52f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f52fa7fc640 1 -- 172.21.15.12:0/2928491545 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0bc070 con 0x7f5304139560 2024-04-27T07:05:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f52fa7fc640 1 -- 172.21.15.12:0/2928491545 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52fc0ad430 con 0x7f5304139560 2024-04-27T07:05:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530bc1e640 1 -- 172.21.15.12:0/2928491545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53040aedd0 con 0x7f5304139560 2024-04-27T07:05:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f52fa7fc640 1 -- 172.21.15.12:0/2928491545 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00a4a0 con 0x7f5304139560 2024-04-27T07:05:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530bc1e640 1 -- 172.21.15.12:0/2928491545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53040b8460 con 0x7f5304139560 2024-04-27T07:05:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f530bc1e640 1 -- 172.21.15.12:0/2928491545 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5304068c20 con 0x7f5304139560 2024-04-27T07:05:05.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f52fa7fc640 1 -- 172.21.15.12:0/2928491545 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f52fc0bf070 con 0x7f5304139560 2024-04-27T07:05:05.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f52fa7fc640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f52e8068380 0x7f52e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:05.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f52fa7fc640 1 -- 172.21.15.12:0/2928491545 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f52fc1207d0 con 0x7f5304139560 2024-04-27T07:05:05.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.820+0000 7f53099ba640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f52e8068380 0x7f52e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:05.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.824+0000 7f53099ba640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f52e8068380 0x7f52e806a840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5300005520 tx=0x7f5300007f20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:05.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.824+0000 7f52fa7fc640 1 -- 172.21.15.12:0/2928491545 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f52fc017070 con 0x7f5304139560 2024-04-27T07:05:05.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.976+0000 7f530bc1e640 1 -- 172.21.15.12:0/2928491545 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f530413ca10 con 0x7f52e8068380 2024-04-27T07:05:06.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:05.996+0000 7f52fa7fc640 1 -- 172.21.15.12:0/2928491545 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251954 (secure 0 0 0) 0x7f530413ca10 con 0x7f52e8068380 2024-04-27T07:05:06.004 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:06.005 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:05:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 -- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f52e8068380 msgr2=0x7f52e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f52e8068380 0x7f52e806a840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5300005520 tx=0x7f5300007f20 comp rx=0 tx=0).stop 2024-04-27T07:05:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 -- 172.21.15.12:0/2928491545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5304139560 msgr2=0x7f53041399e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5304139560 0x7f53041399e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f52fc01cdd0 tx=0x7f52f0005840 comp rx=0 tx=0).stop 2024-04-27T07:05:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 -- 172.21.15.12:0/2928491545 shutdown_connections 2024-04-27T07:05:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f52e8068380 0x7f52e806a840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5304139560 0x7f53041399e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f53040afc20 0x7f53040ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 --2- 172.21.15.12:0/2928491545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5304144070 0x7f53041dab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 -- 172.21.15.12:0/2928491545 >> 172.21.15.12:0/2928491545 conn(0x7f5304067d10 msgr2=0x7f53041cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 -- 172.21.15.12:0/2928491545 shutdown_connections 2024-04-27T07:05:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.000+0000 7f530bc1e640 1 -- 172.21.15.12:0/2928491545 wait complete. 2024-04-27T07:05:06.021 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-04-27T07:05:04.625671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236568,"kb_used_data":3740,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738152,"statfs":{"total":773094113280,"available":772851867648,"internally_reserved":0,"allocated":3829760,"data_stored":2171352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.083351"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577259+0000","last_change":"2024-04-27T07:05:01.619766+0000","last_active":"2024-04-27T07:05:02.577259+0000","last_peered":"2024-04-27T07:05:02.577259+0000","last_clean":"2024-04-27T07:05:02.577259+0000","last_became_active":"2024-04-27T07:05:01.619106+0000","last_became_peered":"2024-04-27T07:05:01.619106+0000","last_unstale":"2024-04-27T07:05:02.577259+0000","last_undegraded":"2024-04-27T07:05:02.577259+0000","last_fullsized":"2024-04-27T07:05:02.577259+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:56:25.278857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903942+0000","last_change":"2024-04-27T07:05:01.590887+0000","last_active":"2024-04-27T07:05:02.903942+0000","last_peered":"2024-04-27T07:05:02.903942+0000","last_clean":"2024-04-27T07:05:02.903942+0000","last_became_active":"2024-04-27T07:05:01.590574+0000","last_became_peered":"2024-04-27T07:05:01.590574+0000","last_unstale":"2024-04-27T07:05:02.903942+0000","last_undegraded":"2024-04-27T07:05:02.903942+0000","last_fullsized":"2024-04-27T07:05:02.903942+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:11.729329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904326+0000","last_change":"2024-04-27T07:05:01.600366+0000","last_active":"2024-04-27T07:05:02.904326+0000","last_peered":"2024-04-27T07:05:02.904326+0000","last_clean":"2024-04-27T07:05:02.904326+0000","last_became_active":"2024-04-27T07:05:01.599448+0000","last_became_peered":"2024-04-27T07:05:01.599448+0000","last_unstale":"2024-04-27T07:05:02.904326+0000","last_undegraded":"2024-04-27T07:05:02.904326+0000","last_fullsized":"2024-04-27T07:05:02.904326+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:22:36.544610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905027+0000","last_change":"2024-04-27T07:05:01.601089+0000","last_active":"2024-04-27T07:05:02.905027+0000","last_peered":"2024-04-27T07:05:02.905027+0000","last_clean":"2024-04-27T07:05:02.905027+0000","last_became_active":"2024-04-27T07:05:01.600833+0000","last_became_peered":"2024-04-27T07:05:01.600833+0000","last_unstale":"2024-04-27T07:05:02.905027+0000","last_undegraded":"2024-04-27T07:05:02.905027+0000","last_fullsized":"2024-04-27T07:05:02.905027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:07:11.751089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576494+0000","last_change":"2024-04-27T07:05:01.596148+0000","last_active":"2024-04-27T07:05:02.576494+0000","last_peered":"2024-04-27T07:05:02.576494+0000","last_clean":"2024-04-27T07:05:02.576494+0000","last_became_active":"2024-04-27T07:05:01.595610+0000","last_became_peered":"2024-04-27T07:05:01.595610+0000","last_unstale":"2024-04-27T07:05:02.576494+0000","last_undegraded":"2024-04-27T07:05:02.576494+0000","last_fullsized":"2024-04-27T07:05:02.576494+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:35:42.870388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903963+0000","last_change":"2024-04-27T07:05:01.612863+0000","last_active":"2024-04-27T07:05:02.903963+0000","last_peered":"2024-04-27T07:05:02.903963+0000","last_clean":"2024-04-27T07:05:02.903963+0000","last_became_active":"2024-04-27T07:05:01.612680+0000","last_became_peered":"2024-04-27T07:05:01.612680+0000","last_unstale":"2024-04-27T07:05:02.903963+0000","last_undegraded":"2024-04-27T07:05:02.903963+0000","last_fullsized":"2024-04-27T07:05:02.903963+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:16:21.551875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902590+0000","last_change":"2024-04-27T07:05:01.619583+0000","last_active":"2024-04-27T07:05:02.902590+0000","last_peered":"2024-04-27T07:05:02.902590+0000","last_clean":"2024-04-27T07:05:02.902590+0000","last_became_active":"2024-04-27T07:05:01.619090+0000","last_became_peered":"2024-04-27T07:05:01.619090+0000","last_unstale":"2024-04-27T07:05:02.902590+0000","last_undegraded":"2024-04-27T07:05:02.902590+0000","last_fullsized":"2024-04-27T07:05:02.902590+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:57:46.218063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902222+0000","last_change":"2024-04-27T07:05:01.619624+0000","last_active":"2024-04-27T07:05:02.902222+0000","last_peered":"2024-04-27T07:05:02.902222+0000","last_clean":"2024-04-27T07:05:02.902222+0000","last_became_active":"2024-04-27T07:05:01.618807+0000","last_became_peered":"2024-04-27T07:05:01.618807+0000","last_unstale":"2024-04-27T07:05:02.902222+0000","last_undegraded":"2024-04-27T07:05:02.902222+0000","last_fullsized":"2024-04-27T07:05:02.902222+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:27:03.887951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577135+0000","last_change":"2024-04-27T07:05:01.593491+0000","last_active":"2024-04-27T07:05:02.577135+0000","last_peered":"2024-04-27T07:05:02.577135+0000","last_clean":"2024-04-27T07:05:02.577135+0000","last_became_active":"2024-04-27T07:05:01.592754+0000","last_became_peered":"2024-04-27T07:05:01.592754+0000","last_unstale":"2024-04-27T07:05:02.577135+0000","last_undegraded":"2024-04-27T07:05:02.577135+0000","last_fullsized":"2024-04-27T07:05:02.577135+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:07:11.751089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906136+0000","last_change":"2024-04-27T07:05:01.596304+0000","last_active":"2024-04-27T07:05:02.906136+0000","last_peered":"2024-04-27T07:05:02.906136+0000","last_clean":"2024-04-27T07:05:02.906136+0000","last_became_active":"2024-04-27T07:05:01.595898+0000","last_became_peered":"2024-04-27T07:05:01.595898+0000","last_unstale":"2024-04-27T07:05:02.906136+0000","last_undegraded":"2024-04-27T07:05:02.906136+0000","last_fullsized":"2024-04-27T07:05:02.906136+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:45:14.410767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902886+0000","last_change":"2024-04-27T07:05:01.615052+0000","last_active":"2024-04-27T07:05:02.902886+0000","last_peered":"2024-04-27T07:05:02.902886+0000","last_clean":"2024-04-27T07:05:02.902886+0000","last_became_active":"2024-04-27T07:05:01.614397+0000","last_became_peered":"2024-04-27T07:05:01.614397+0000","last_unstale":"2024-04-27T07:05:02.902886+0000","last_undegraded":"2024-04-27T07:05:02.902886+0000","last_fullsized":"2024-04-27T07:05:02.902886+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:57:13.964316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903764+0000","last_change":"2024-04-27T07:05:01.611400+0000","last_active":"2024-04-27T07:05:02.903764+0000","last_peered":"2024-04-27T07:05:02.903764+0000","last_clean":"2024-04-27T07:05:02.903764+0000","last_became_active":"2024-04-27T07:05:01.610436+0000","last_became_peered":"2024-04-27T07:05:01.610436+0000","last_unstale":"2024-04-27T07:05:02.903764+0000","last_undegraded":"2024-04-27T07:05:02.903764+0000","last_fullsized":"2024-04-27T07:05:02.903764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:11.729329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576774+0000","last_change":"2024-04-27T07:05:01.589627+0000","last_active":"2024-04-27T07:05:02.576774+0000","last_peered":"2024-04-27T07:05:02.576774+0000","last_clean":"2024-04-27T07:05:02.576774+0000","last_became_active":"2024-04-27T07:05:01.589376+0000","last_became_peered":"2024-04-27T07:05:01.589376+0000","last_unstale":"2024-04-27T07:05:02.576774+0000","last_undegraded":"2024-04-27T07:05:02.576774+0000","last_fullsized":"2024-04-27T07:05:02.576774+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:11:15.532969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904361+0000","last_change":"2024-04-27T07:05:01.597525+0000","last_active":"2024-04-27T07:05:02.904361+0000","last_peered":"2024-04-27T07:05:02.904361+0000","last_clean":"2024-04-27T07:05:02.904361+0000","last_became_active":"2024-04-27T07:05:01.596969+0000","last_became_peered":"2024-04-27T07:05:01.596969+0000","last_unstale":"2024-04-27T07:05:02.904361+0000","last_undegraded":"2024-04-27T07:05:02.904361+0000","last_fullsized":"2024-04-27T07:05:02.904361+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:57:13.964316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903342+0000","last_change":"2024-04-27T07:05:01.615038+0000","last_active":"2024-04-27T07:05:02.903342+0000","last_peered":"2024-04-27T07:05:02.903342+0000","last_clean":"2024-04-27T07:05:02.903342+0000","last_became_active":"2024-04-27T07:05:01.614661+0000","last_became_peered":"2024-04-27T07:05:01.614661+0000","last_unstale":"2024-04-27T07:05:02.903342+0000","last_undegraded":"2024-04-27T07:05:02.903342+0000","last_fullsized":"2024-04-27T07:05:02.903342+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:27:03.887951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903121+0000","last_change":"2024-04-27T07:05:01.625925+0000","last_active":"2024-04-27T07:05:02.903121+0000","last_peered":"2024-04-27T07:05:02.903121+0000","last_clean":"2024-04-27T07:05:02.903121+0000","last_became_active":"2024-04-27T07:05:01.625265+0000","last_became_peered":"2024-04-27T07:05:01.625265+0000","last_unstale":"2024-04-27T07:05:02.903121+0000","last_undegraded":"2024-04-27T07:05:02.903121+0000","last_fullsized":"2024-04-27T07:05:02.903121+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:43:55.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903799+0000","last_change":"2024-04-27T07:05:01.620564+0000","last_active":"2024-04-27T07:05:02.903799+0000","last_peered":"2024-04-27T07:05:02.903799+0000","last_clean":"2024-04-27T07:05:02.903799+0000","last_became_active":"2024-04-27T07:05:01.620060+0000","last_became_peered":"2024-04-27T07:05:01.620060+0000","last_unstale":"2024-04-27T07:05:02.903799+0000","last_undegraded":"2024-04-27T07:05:02.903799+0000","last_fullsized":"2024-04-27T07:05:02.903799+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:54:27.931459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904353+0000","last_change":"2024-04-27T07:05:01.595065+0000","last_active":"2024-04-27T07:05:02.904353+0000","last_peered":"2024-04-27T07:05:02.904353+0000","last_clean":"2024-04-27T07:05:02.904353+0000","last_became_active":"2024-04-27T07:05:01.594768+0000","last_became_peered":"2024-04-27T07:05:01.594768+0000","last_unstale":"2024-04-27T07:05:02.904353+0000","last_undegraded":"2024-04-27T07:05:02.904353+0000","last_fullsized":"2024-04-27T07:05:02.904353+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:22:36.544610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904007+0000","last_change":"2024-04-27T07:05:01.596597+0000","last_active":"2024-04-27T07:05:02.904007+0000","last_peered":"2024-04-27T07:05:02.904007+0000","last_clean":"2024-04-27T07:05:02.904007+0000","last_became_active":"2024-04-27T07:05:01.594969+0000","last_became_peered":"2024-04-27T07:05:01.594969+0000","last_unstale":"2024-04-27T07:05:02.904007+0000","last_undegraded":"2024-04-27T07:05:02.904007+0000","last_fullsized":"2024-04-27T07:05:02.904007+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:57:46.218063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576307+0000","last_change":"2024-04-27T07:05:01.612692+0000","last_active":"2024-04-27T07:05:02.576307+0000","last_peered":"2024-04-27T07:05:02.576307+0000","last_clean":"2024-04-27T07:05:02.576307+0000","last_became_active":"2024-04-27T07:05:01.612263+0000","last_became_peered":"2024-04-27T07:05:01.612263+0000","last_unstale":"2024-04-27T07:05:02.576307+0000","last_undegraded":"2024-04-27T07:05:02.576307+0000","last_fullsized":"2024-04-27T07:05:02.576307+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:07:53.287473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574899+0000","last_change":"2024-04-27T07:05:01.579406+0000","last_active":"2024-04-27T07:05:02.574899+0000","last_peered":"2024-04-27T07:05:02.574899+0000","last_clean":"2024-04-27T07:05:02.574899+0000","last_became_active":"2024-04-27T07:05:01.578630+0000","last_became_peered":"2024-04-27T07:05:01.578630+0000","last_unstale":"2024-04-27T07:05:02.574899+0000","last_undegraded":"2024-04-27T07:05:02.574899+0000","last_fullsized":"2024-04-27T07:05:02.574899+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:33:20.122035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574752+0000","last_change":"2024-04-27T07:05:01.601560+0000","last_active":"2024-04-27T07:05:02.574752+0000","last_peered":"2024-04-27T07:05:02.574752+0000","last_clean":"2024-04-27T07:05:02.574752+0000","last_became_active":"2024-04-27T07:05:01.600159+0000","last_became_peered":"2024-04-27T07:05:01.600159+0000","last_unstale":"2024-04-27T07:05:02.574752+0000","last_undegraded":"2024-04-27T07:05:02.574752+0000","last_fullsized":"2024-04-27T07:05:02.574752+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:41:35.033852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903224+0000","last_change":"2024-04-27T07:05:01.610220+0000","last_active":"2024-04-27T07:05:02.903224+0000","last_peered":"2024-04-27T07:05:02.903224+0000","last_clean":"2024-04-27T07:05:02.903224+0000","last_became_active":"2024-04-27T07:05:01.609992+0000","last_became_peered":"2024-04-27T07:05:01.609992+0000","last_unstale":"2024-04-27T07:05:02.903224+0000","last_undegraded":"2024-04-27T07:05:02.903224+0000","last_fullsized":"2024-04-27T07:05:02.903224+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:41:35.033852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903106+0000","last_change":"2024-04-27T07:05:01.597341+0000","last_active":"2024-04-27T07:05:02.903106+0000","last_peered":"2024-04-27T07:05:02.903106+0000","last_clean":"2024-04-27T07:05:02.903106+0000","last_became_active":"2024-04-27T07:05:01.596539+0000","last_became_peered":"2024-04-27T07:05:01.596539+0000","last_unstale":"2024-04-27T07:05:02.903106+0000","last_undegraded":"2024-04-27T07:05:02.903106+0000","last_fullsized":"2024-04-27T07:05:02.903106+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:50.115271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905837+0000","last_change":"2024-04-27T07:05:01.616680+0000","last_active":"2024-04-27T07:05:02.905837+0000","last_peered":"2024-04-27T07:05:02.905837+0000","last_clean":"2024-04-27T07:05:02.905837+0000","last_became_active":"2024-04-27T07:05:01.616383+0000","last_became_peered":"2024-04-27T07:05:01.616383+0000","last_unstale":"2024-04-27T07:05:02.905837+0000","last_undegraded":"2024-04-27T07:05:02.905837+0000","last_fullsized":"2024-04-27T07:05:02.905837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:22.396865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904732+0000","last_change":"2024-04-27T07:05:01.610200+0000","last_active":"2024-04-27T07:05:02.904732+0000","last_peered":"2024-04-27T07:05:02.904732+0000","last_clean":"2024-04-27T07:05:02.904732+0000","last_became_active":"2024-04-27T07:05:01.609866+0000","last_became_peered":"2024-04-27T07:05:01.609866+0000","last_unstale":"2024-04-27T07:05:02.904732+0000","last_undegraded":"2024-04-27T07:05:02.904732+0000","last_fullsized":"2024-04-27T07:05:02.904732+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:43:55.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.907760+0000","last_change":"2024-04-27T07:05:01.619276+0000","last_active":"2024-04-27T07:05:02.907760+0000","last_peered":"2024-04-27T07:05:02.907760+0000","last_clean":"2024-04-27T07:05:02.907760+0000","last_became_active":"2024-04-27T07:05:01.618963+0000","last_became_peered":"2024-04-27T07:05:01.618963+0000","last_unstale":"2024-04-27T07:05:02.907760+0000","last_undegraded":"2024-04-27T07:05:02.907760+0000","last_fullsized":"2024-04-27T07:05:02.907760+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:25:16.721450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904178+0000","last_change":"2024-04-27T07:05:01.596015+0000","last_active":"2024-04-27T07:05:02.904178+0000","last_peered":"2024-04-27T07:05:02.904178+0000","last_clean":"2024-04-27T07:05:02.904178+0000","last_became_active":"2024-04-27T07:05:01.595230+0000","last_became_peered":"2024-04-27T07:05:01.595230+0000","last_unstale":"2024-04-27T07:05:02.904178+0000","last_undegraded":"2024-04-27T07:05:02.904178+0000","last_fullsized":"2024-04-27T07:05:02.904178+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:25:52.050396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575170+0000","last_change":"2024-04-27T07:05:01.602051+0000","last_active":"2024-04-27T07:05:02.575170+0000","last_peered":"2024-04-27T07:05:02.575170+0000","last_clean":"2024-04-27T07:05:02.575170+0000","last_became_active":"2024-04-27T07:05:01.600826+0000","last_became_peered":"2024-04-27T07:05:01.600826+0000","last_unstale":"2024-04-27T07:05:02.575170+0000","last_undegraded":"2024-04-27T07:05:02.575170+0000","last_fullsized":"2024-04-27T07:05:02.575170+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:21.852593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904746+0000","last_change":"2024-04-27T07:05:01.619580+0000","last_active":"2024-04-27T07:05:02.904746+0000","last_peered":"2024-04-27T07:05:02.904746+0000","last_clean":"2024-04-27T07:05:02.904746+0000","last_became_active":"2024-04-27T07:05:01.618840+0000","last_became_peered":"2024-04-27T07:05:01.618840+0000","last_unstale":"2024-04-27T07:05:02.904746+0000","last_undegraded":"2024-04-27T07:05:02.904746+0000","last_fullsized":"2024-04-27T07:05:02.904746+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:54.205836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576911+0000","last_change":"2024-04-27T07:05:01.611749+0000","last_active":"2024-04-27T07:05:02.576911+0000","last_peered":"2024-04-27T07:05:02.576911+0000","last_clean":"2024-04-27T07:05:02.576911+0000","last_became_active":"2024-04-27T07:05:01.610895+0000","last_became_peered":"2024-04-27T07:05:01.610895+0000","last_unstale":"2024-04-27T07:05:02.576911+0000","last_undegraded":"2024-04-27T07:05:02.576911+0000","last_fullsized":"2024-04-27T07:05:02.576911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:44.123073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902226+0000","last_change":"2024-04-27T07:05:01.630879+0000","last_active":"2024-04-27T07:05:02.902226+0000","last_peered":"2024-04-27T07:05:02.902226+0000","last_clean":"2024-04-27T07:05:02.902226+0000","last_became_active":"2024-04-27T07:05:01.630279+0000","last_became_peered":"2024-04-27T07:05:01.630279+0000","last_unstale":"2024-04-27T07:05:02.902226+0000","last_undegraded":"2024-04-27T07:05:02.902226+0000","last_fullsized":"2024-04-27T07:05:02.902226+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:11.729329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906808+0000","last_change":"2024-04-27T07:05:01.610677+0000","last_active":"2024-04-27T07:05:02.906808+0000","last_peered":"2024-04-27T07:05:02.906808+0000","last_clean":"2024-04-27T07:05:02.906808+0000","last_became_active":"2024-04-27T07:05:01.610473+0000","last_became_peered":"2024-04-27T07:05:01.610473+0000","last_unstale":"2024-04-27T07:05:02.906808+0000","last_undegraded":"2024-04-27T07:05:02.906808+0000","last_fullsized":"2024-04-27T07:05:02.906808+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:24.705960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"28'4","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577957+0000","last_change":"2024-04-27T07:05:00.576697+0000","last_active":"2024-04-27T07:05:02.577957+0000","last_peered":"2024-04-27T07:05:02.577957+0000","last_clean":"2024-04-27T07:05:02.577957+0000","last_became_active":"2024-04-27T07:05:00.576468+0000","last_became_peered":"2024-04-27T07:05:00.576468+0000","last_unstale":"2024-04-27T07:05:02.577957+0000","last_undegraded":"2024-04-27T07:05:02.577957+0000","last_fullsized":"2024-04-27T07:05:02.577957+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:02.662003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577665+0000","last_change":"2024-04-27T07:05:01.582659+0000","last_active":"2024-04-27T07:05:02.577665+0000","last_peered":"2024-04-27T07:05:02.577665+0000","last_clean":"2024-04-27T07:05:02.577665+0000","last_became_active":"2024-04-27T07:05:01.581933+0000","last_became_peered":"2024-04-27T07:05:01.581933+0000","last_unstale":"2024-04-27T07:05:02.577665+0000","last_undegraded":"2024-04-27T07:05:02.577665+0000","last_fullsized":"2024-04-27T07:05:02.577665+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:41:35.033852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906735+0000","last_change":"2024-04-27T07:05:00.568249+0000","last_active":"2024-04-27T07:05:02.906735+0000","last_peered":"2024-04-27T07:05:02.906735+0000","last_clean":"2024-04-27T07:05:02.906735+0000","last_became_active":"2024-04-27T07:05:00.567955+0000","last_became_peered":"2024-04-27T07:05:00.567955+0000","last_unstale":"2024-04-27T07:05:02.906735+0000","last_undegraded":"2024-04-27T07:05:02.906735+0000","last_fullsized":"2024-04-27T07:05:02.906735+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:39.842835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575425+0000","last_change":"2024-04-27T07:05:01.603708+0000","last_active":"2024-04-27T07:05:02.575425+0000","last_peered":"2024-04-27T07:05:02.575425+0000","last_clean":"2024-04-27T07:05:02.575425+0000","last_became_active":"2024-04-27T07:05:01.602794+0000","last_became_peered":"2024-04-27T07:05:01.602794+0000","last_unstale":"2024-04-27T07:05:02.575425+0000","last_undegraded":"2024-04-27T07:05:02.575425+0000","last_fullsized":"2024-04-27T07:05:02.575425+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:24.705960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904301+0000","last_change":"2024-04-27T07:05:00.580459+0000","last_active":"2024-04-27T07:05:02.904301+0000","last_peered":"2024-04-27T07:05:02.904301+0000","last_clean":"2024-04-27T07:05:02.904301+0000","last_became_active":"2024-04-27T07:05:00.579931+0000","last_became_peered":"2024-04-27T07:05:00.579931+0000","last_unstale":"2024-04-27T07:05:02.904301+0000","last_undegraded":"2024-04-27T07:05:02.904301+0000","last_fullsized":"2024-04-27T07:05:02.904301+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:43.113828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.907636+0000","last_change":"2024-04-27T07:05:01.601059+0000","last_active":"2024-04-27T07:05:02.907636+0000","last_peered":"2024-04-27T07:05:02.907636+0000","last_clean":"2024-04-27T07:05:02.907636+0000","last_became_active":"2024-04-27T07:05:01.600683+0000","last_became_peered":"2024-04-27T07:05:01.600683+0000","last_unstale":"2024-04-27T07:05:02.907636+0000","last_undegraded":"2024-04-27T07:05:02.907636+0000","last_fullsized":"2024-04-27T07:05:02.907636+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:50:18.189750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576072+0000","last_change":"2024-04-27T07:05:00.564610+0000","last_active":"2024-04-27T07:05:02.576072+0000","last_peered":"2024-04-27T07:05:02.576072+0000","last_clean":"2024-04-27T07:05:02.576072+0000","last_became_active":"2024-04-27T07:05:00.564183+0000","last_became_peered":"2024-04-27T07:05:00.564183+0000","last_unstale":"2024-04-27T07:05:02.576072+0000","last_undegraded":"2024-04-27T07:05:02.576072+0000","last_fullsized":"2024-04-27T07:05:02.576072+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:39:49.203139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904960+0000","last_change":"2024-04-27T07:05:01.626297+0000","last_active":"2024-04-27T07:05:02.904960+0000","last_peered":"2024-04-27T07:05:02.904960+0000","last_clean":"2024-04-27T07:05:02.904960+0000","last_became_active":"2024-04-27T07:05:01.625643+0000","last_became_peered":"2024-04-27T07:05:01.625643+0000","last_unstale":"2024-04-27T07:05:02.904960+0000","last_undegraded":"2024-04-27T07:05:02.904960+0000","last_fullsized":"2024-04-27T07:05:02.904960+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:35.493213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575183+0000","last_change":"2024-04-27T07:05:00.580618+0000","last_active":"2024-04-27T07:05:02.575183+0000","last_peered":"2024-04-27T07:05:02.575183+0000","last_clean":"2024-04-27T07:05:02.575183+0000","last_became_active":"2024-04-27T07:05:00.579811+0000","last_became_peered":"2024-04-27T07:05:00.579811+0000","last_unstale":"2024-04-27T07:05:02.575183+0000","last_undegraded":"2024-04-27T07:05:02.575183+0000","last_fullsized":"2024-04-27T07:05:02.575183+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:03:30.967635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903759+0000","last_change":"2024-04-27T07:05:01.609943+0000","last_active":"2024-04-27T07:05:02.903759+0000","last_peered":"2024-04-27T07:05:02.903759+0000","last_clean":"2024-04-27T07:05:02.903759+0000","last_became_active":"2024-04-27T07:05:01.608942+0000","last_became_peered":"2024-04-27T07:05:01.608942+0000","last_unstale":"2024-04-27T07:05:02.903759+0000","last_undegraded":"2024-04-27T07:05:02.903759+0000","last_fullsized":"2024-04-27T07:05:02.903759+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:16:21.551875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574662+0000","last_change":"2024-04-27T07:05:00.583325+0000","last_active":"2024-04-27T07:05:02.574662+0000","last_peered":"2024-04-27T07:05:02.574662+0000","last_clean":"2024-04-27T07:05:02.574662+0000","last_became_active":"2024-04-27T07:05:00.581179+0000","last_became_peered":"2024-04-27T07:05:00.581179+0000","last_unstale":"2024-04-27T07:05:02.574662+0000","last_undegraded":"2024-04-27T07:05:02.574662+0000","last_fullsized":"2024-04-27T07:05:02.574662+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:04:27.277283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574396+0000","last_change":"2024-04-27T07:05:01.584737+0000","last_active":"2024-04-27T07:05:02.574396+0000","last_peered":"2024-04-27T07:05:02.574396+0000","last_clean":"2024-04-27T07:05:02.574396+0000","last_became_active":"2024-04-27T07:05:01.584001+0000","last_became_peered":"2024-04-27T07:05:01.584001+0000","last_unstale":"2024-04-27T07:05:02.574396+0000","last_undegraded":"2024-04-27T07:05:02.574396+0000","last_fullsized":"2024-04-27T07:05:02.574396+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:56:25.278857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902799+0000","last_change":"2024-04-27T07:05:00.576227+0000","last_active":"2024-04-27T07:05:02.902799+0000","last_peered":"2024-04-27T07:05:02.902799+0000","last_clean":"2024-04-27T07:05:02.902799+0000","last_became_active":"2024-04-27T07:05:00.575365+0000","last_became_peered":"2024-04-27T07:05:00.575365+0000","last_unstale":"2024-04-27T07:05:02.902799+0000","last_undegraded":"2024-04-27T07:05:02.902799+0000","last_fullsized":"2024-04-27T07:05:02.902799+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:24:20.674931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904849+0000","last_change":"2024-04-27T07:05:01.613711+0000","last_active":"2024-04-27T07:05:02.904849+0000","last_peered":"2024-04-27T07:05:02.904849+0000","last_clean":"2024-04-27T07:05:02.904849+0000","last_became_active":"2024-04-27T07:05:01.613373+0000","last_became_peered":"2024-04-27T07:05:01.613373+0000","last_unstale":"2024-04-27T07:05:02.904849+0000","last_undegraded":"2024-04-27T07:05:02.904849+0000","last_fullsized":"2024-04-27T07:05:02.904849+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:09:25.577049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574234+0000","last_change":"2024-04-27T07:05:00.566993+0000","last_active":"2024-04-27T07:05:02.574234+0000","last_peered":"2024-04-27T07:05:02.574234+0000","last_clean":"2024-04-27T07:05:02.574234+0000","last_became_active":"2024-04-27T07:05:00.566758+0000","last_became_peered":"2024-04-27T07:05:00.566758+0000","last_unstale":"2024-04-27T07:05:02.574234+0000","last_undegraded":"2024-04-27T07:05:02.574234+0000","last_fullsized":"2024-04-27T07:05:02.574234+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:02.662003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905468+0000","last_change":"2024-04-27T07:05:01.597697+0000","last_active":"2024-04-27T07:05:02.905468+0000","last_peered":"2024-04-27T07:05:02.905468+0000","last_clean":"2024-04-27T07:05:02.905468+0000","last_became_active":"2024-04-27T07:05:01.596910+0000","last_became_peered":"2024-04-27T07:05:01.596910+0000","last_unstale":"2024-04-27T07:05:02.905468+0000","last_undegraded":"2024-04-27T07:05:02.905468+0000","last_fullsized":"2024-04-27T07:05:02.905468+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:57:13.964316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906248+0000","last_change":"2024-04-27T07:05:00.584576+0000","last_active":"2024-04-27T07:05:02.906248+0000","last_peered":"2024-04-27T07:05:02.906248+0000","last_clean":"2024-04-27T07:05:02.906248+0000","last_became_active":"2024-04-27T07:05:00.584108+0000","last_became_peered":"2024-04-27T07:05:00.584108+0000","last_unstale":"2024-04-27T07:05:02.906248+0000","last_undegraded":"2024-04-27T07:05:02.906248+0000","last_fullsized":"2024-04-27T07:05:02.906248+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:41:34.024607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906001+0000","last_change":"2024-04-27T07:05:01.595435+0000","last_active":"2024-04-27T07:05:02.906001+0000","last_peered":"2024-04-27T07:05:02.906001+0000","last_clean":"2024-04-27T07:05:02.906001+0000","last_became_active":"2024-04-27T07:05:01.594792+0000","last_became_peered":"2024-04-27T07:05:01.594792+0000","last_unstale":"2024-04-27T07:05:02.906001+0000","last_undegraded":"2024-04-27T07:05:02.906001+0000","last_fullsized":"2024-04-27T07:05:02.906001+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:24:54.205836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906884+0000","last_change":"2024-04-27T07:05:00.573881+0000","last_active":"2024-04-27T07:05:02.906884+0000","last_peered":"2024-04-27T07:05:02.906884+0000","last_clean":"2024-04-27T07:05:02.906884+0000","last_became_active":"2024-04-27T07:05:00.573338+0000","last_became_peered":"2024-04-27T07:05:00.573338+0000","last_unstale":"2024-04-27T07:05:02.906884+0000","last_undegraded":"2024-04-27T07:05:02.906884+0000","last_fullsized":"2024-04-27T07:05:02.906884+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:22.266250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904246+0000","last_change":"2024-04-27T07:05:01.610120+0000","last_active":"2024-04-27T07:05:02.904246+0000","last_peered":"2024-04-27T07:05:02.904246+0000","last_clean":"2024-04-27T07:05:02.904246+0000","last_became_active":"2024-04-27T07:05:01.609805+0000","last_became_peered":"2024-04-27T07:05:01.609805+0000","last_unstale":"2024-04-27T07:05:02.904246+0000","last_undegraded":"2024-04-27T07:05:02.904246+0000","last_fullsized":"2024-04-27T07:05:02.904246+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:25:52.050396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"28'3","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904476+0000","last_change":"2024-04-27T07:05:00.574817+0000","last_active":"2024-04-27T07:05:02.904476+0000","last_peered":"2024-04-27T07:05:02.904476+0000","last_clean":"2024-04-27T07:05:02.904476+0000","last_became_active":"2024-04-27T07:05:00.573530+0000","last_became_peered":"2024-04-27T07:05:00.573530+0000","last_unstale":"2024-04-27T07:05:02.904476+0000","last_undegraded":"2024-04-27T07:05:02.904476+0000","last_fullsized":"2024-04-27T07:05:02.904476+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:33:19.112790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904123+0000","last_change":"2024-04-27T07:05:01.609943+0000","last_active":"2024-04-27T07:05:02.904123+0000","last_peered":"2024-04-27T07:05:02.904123+0000","last_clean":"2024-04-27T07:05:02.904123+0000","last_became_active":"2024-04-27T07:05:01.609179+0000","last_became_peered":"2024-04-27T07:05:01.609179+0000","last_unstale":"2024-04-27T07:05:02.904123+0000","last_undegraded":"2024-04-27T07:05:02.904123+0000","last_fullsized":"2024-04-27T07:05:02.904123+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:57:46.218063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"28'6","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904519+0000","last_change":"2024-04-27T07:05:00.581950+0000","last_active":"2024-04-27T07:05:02.904519+0000","last_peered":"2024-04-27T07:05:02.904519+0000","last_clean":"2024-04-27T07:05:02.904519+0000","last_became_active":"2024-04-27T07:05:00.581540+0000","last_became_peered":"2024-04-27T07:05:00.581540+0000","last_unstale":"2024-04-27T07:05:02.904519+0000","last_undegraded":"2024-04-27T07:05:02.904519+0000","last_fullsized":"2024-04-27T07:05:02.904519+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:33:19.112790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904841+0000","last_change":"2024-04-27T07:05:01.613661+0000","last_active":"2024-04-27T07:05:02.904841+0000","last_peered":"2024-04-27T07:05:02.904841+0000","last_clean":"2024-04-27T07:05:02.904841+0000","last_became_active":"2024-04-27T07:05:01.613346+0000","last_became_peered":"2024-04-27T07:05:01.613346+0000","last_unstale":"2024-04-27T07:05:02.904841+0000","last_undegraded":"2024-04-27T07:05:02.904841+0000","last_fullsized":"2024-04-27T07:05:02.904841+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:16:24.705960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575383+0000","last_change":"2024-04-27T07:05:00.563391+0000","last_active":"2024-04-27T07:05:02.575383+0000","last_peered":"2024-04-27T07:05:02.575383+0000","last_clean":"2024-04-27T07:05:02.575383+0000","last_became_active":"2024-04-27T07:05:00.563127+0000","last_became_peered":"2024-04-27T07:05:00.563127+0000","last_unstale":"2024-04-27T07:05:02.575383+0000","last_undegraded":"2024-04-27T07:05:02.575383+0000","last_fullsized":"2024-04-27T07:05:02.575383+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:53:39.852098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905410+0000","last_change":"2024-04-27T07:05:01.583757+0000","last_active":"2024-04-27T07:05:02.905410+0000","last_peered":"2024-04-27T07:05:02.905410+0000","last_clean":"2024-04-27T07:05:02.905410+0000","last_became_active":"2024-04-27T07:05:01.583487+0000","last_became_peered":"2024-04-27T07:05:01.583487+0000","last_unstale":"2024-04-27T07:05:02.905410+0000","last_undegraded":"2024-04-27T07:05:02.905410+0000","last_fullsized":"2024-04-27T07:05:02.905410+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:54:27.931459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905265+0000","last_change":"2024-04-27T07:05:00.578652+0000","last_active":"2024-04-27T07:05:02.905265+0000","last_peered":"2024-04-27T07:05:02.905265+0000","last_clean":"2024-04-27T07:05:02.905265+0000","last_became_active":"2024-04-27T07:05:00.577937+0000","last_became_peered":"2024-04-27T07:05:00.577937+0000","last_unstale":"2024-04-27T07:05:02.905265+0000","last_undegraded":"2024-04-27T07:05:02.905265+0000","last_fullsized":"2024-04-27T07:05:02.905265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:39.842835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902661+0000","last_change":"2024-04-27T07:05:01.626608+0000","last_active":"2024-04-27T07:05:02.902661+0000","last_peered":"2024-04-27T07:05:02.902661+0000","last_clean":"2024-04-27T07:05:02.902661+0000","last_became_active":"2024-04-27T07:05:01.626030+0000","last_became_peered":"2024-04-27T07:05:01.626030+0000","last_unstale":"2024-04-27T07:05:02.902661+0000","last_undegraded":"2024-04-27T07:05:02.902661+0000","last_fullsized":"2024-04-27T07:05:02.902661+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:25:52.050396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903051+0000","last_change":"2024-04-27T07:05:00.582663+0000","last_active":"2024-04-27T07:05:02.903051+0000","last_peered":"2024-04-27T07:05:02.903051+0000","last_clean":"2024-04-27T07:05:02.903051+0000","last_became_active":"2024-04-27T07:05:00.582130+0000","last_became_peered":"2024-04-27T07:05:00.582130+0000","last_unstale":"2024-04-27T07:05:02.903051+0000","last_undegraded":"2024-04-27T07:05:02.903051+0000","last_fullsized":"2024-04-27T07:05:02.903051+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:35:03.637726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577385+0000","last_change":"2024-04-27T07:05:01.611934+0000","last_active":"2024-04-27T07:05:02.577385+0000","last_peered":"2024-04-27T07:05:02.577385+0000","last_clean":"2024-04-27T07:05:02.577385+0000","last_became_active":"2024-04-27T07:05:01.611104+0000","last_became_peered":"2024-04-27T07:05:01.611104+0000","last_unstale":"2024-04-27T07:05:02.577385+0000","last_undegraded":"2024-04-27T07:05:02.577385+0000","last_fullsized":"2024-04-27T07:05:02.577385+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:04:17.124168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"28'8","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903216+0000","last_change":"2024-04-27T07:05:00.582401+0000","last_active":"2024-04-27T07:05:02.903216+0000","last_peered":"2024-04-27T07:05:02.903216+0000","last_clean":"2024-04-27T07:05:02.903216+0000","last_became_active":"2024-04-27T07:05:00.582059+0000","last_became_peered":"2024-04-27T07:05:00.582059+0000","last_unstale":"2024-04-27T07:05:02.903216+0000","last_undegraded":"2024-04-27T07:05:02.903216+0000","last_fullsized":"2024-04-27T07:05:02.903216+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:56:24.269613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576437+0000","last_change":"2024-04-27T07:05:01.624915+0000","last_active":"2024-04-27T07:05:02.576437+0000","last_peered":"2024-04-27T07:05:02.576437+0000","last_clean":"2024-04-27T07:05:02.576437+0000","last_became_active":"2024-04-27T07:05:01.624142+0000","last_became_peered":"2024-04-27T07:05:01.624142+0000","last_unstale":"2024-04-27T07:05:02.576437+0000","last_undegraded":"2024-04-27T07:05:02.576437+0000","last_fullsized":"2024-04-27T07:05:02.576437+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:09:25.577049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906001+0000","last_change":"2024-04-27T07:05:00.580754+0000","last_active":"2024-04-27T07:05:02.906001+0000","last_peered":"2024-04-27T07:05:02.906001+0000","last_clean":"2024-04-27T07:05:02.906001+0000","last_became_active":"2024-04-27T07:05:00.579170+0000","last_became_peered":"2024-04-27T07:05:00.579170+0000","last_unstale":"2024-04-27T07:05:02.906001+0000","last_undegraded":"2024-04-27T07:05:02.906001+0000","last_fullsized":"2024-04-27T07:05:02.906001+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:36:50.917879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905657+0000","last_change":"2024-04-27T07:05:01.598762+0000","last_active":"2024-04-27T07:05:02.905657+0000","last_peered":"2024-04-27T07:05:02.905657+0000","last_clean":"2024-04-27T07:05:02.905657+0000","last_became_active":"2024-04-27T07:05:01.596478+0000","last_became_peered":"2024-04-27T07:05:01.596478+0000","last_unstale":"2024-04-27T07:05:02.905657+0000","last_undegraded":"2024-04-27T07:05:02.905657+0000","last_fullsized":"2024-04-27T07:05:02.905657+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:54:27.931459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576485+0000","last_change":"2024-04-27T07:05:00.581490+0000","last_active":"2024-04-27T07:05:02.576485+0000","last_peered":"2024-04-27T07:05:02.576485+0000","last_clean":"2024-04-27T07:05:02.576485+0000","last_became_active":"2024-04-27T07:05:00.581200+0000","last_became_peered":"2024-04-27T07:05:00.581200+0000","last_unstale":"2024-04-27T07:05:02.576485+0000","last_undegraded":"2024-04-27T07:05:02.576485+0000","last_fullsized":"2024-04-27T07:05:02.576485+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:11:14.523724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905429+0000","last_change":"2024-04-27T07:05:01.600930+0000","last_active":"2024-04-27T07:05:02.905429+0000","last_peered":"2024-04-27T07:05:02.905429+0000","last_clean":"2024-04-27T07:05:02.905429+0000","last_became_active":"2024-04-27T07:05:01.600645+0000","last_became_peered":"2024-04-27T07:05:01.600645+0000","last_unstale":"2024-04-27T07:05:02.905429+0000","last_undegraded":"2024-04-27T07:05:02.905429+0000","last_fullsized":"2024-04-27T07:05:02.905429+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:53:54.641089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576295+0000","last_change":"2024-04-27T07:05:00.563606+0000","last_active":"2024-04-27T07:05:02.576295+0000","last_peered":"2024-04-27T07:05:02.576295+0000","last_clean":"2024-04-27T07:05:02.576295+0000","last_became_active":"2024-04-27T07:05:00.563066+0000","last_became_peered":"2024-04-27T07:05:00.563066+0000","last_unstale":"2024-04-27T07:05:02.576295+0000","last_undegraded":"2024-04-27T07:05:02.576295+0000","last_fullsized":"2024-04-27T07:05:02.576295+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:46.697491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576602+0000","last_change":"2024-04-27T07:05:01.609588+0000","last_active":"2024-04-27T07:05:02.576602+0000","last_peered":"2024-04-27T07:05:02.576602+0000","last_clean":"2024-04-27T07:05:02.576602+0000","last_became_active":"2024-04-27T07:05:01.609314+0000","last_became_peered":"2024-04-27T07:05:01.609314+0000","last_unstale":"2024-04-27T07:05:02.576602+0000","last_undegraded":"2024-04-27T07:05:02.576602+0000","last_fullsized":"2024-04-27T07:05:02.576602+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:53:40.861342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575570+0000","last_change":"2024-04-27T07:05:00.579714+0000","last_active":"2024-04-27T07:05:02.575570+0000","last_peered":"2024-04-27T07:05:02.575570+0000","last_clean":"2024-04-27T07:05:02.575570+0000","last_became_active":"2024-04-27T07:05:00.579470+0000","last_became_peered":"2024-04-27T07:05:00.579470+0000","last_unstale":"2024-04-27T07:05:02.575570+0000","last_undegraded":"2024-04-27T07:05:02.575570+0000","last_fullsized":"2024-04-27T07:05:02.575570+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:07:52.278228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576060+0000","last_change":"2024-04-27T07:05:01.589744+0000","last_active":"2024-04-27T07:05:02.576060+0000","last_peered":"2024-04-27T07:05:02.576060+0000","last_clean":"2024-04-27T07:05:02.576060+0000","last_became_active":"2024-04-27T07:05:01.586706+0000","last_became_peered":"2024-04-27T07:05:01.586706+0000","last_unstale":"2024-04-27T07:05:02.576060+0000","last_undegraded":"2024-04-27T07:05:02.576060+0000","last_fullsized":"2024-04-27T07:05:02.576060+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:47.706736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904096+0000","last_change":"2024-04-27T07:05:00.564682+0000","last_active":"2024-04-27T07:05:02.904096+0000","last_peered":"2024-04-27T07:05:02.904096+0000","last_clean":"2024-04-27T07:05:02.904096+0000","last_became_active":"2024-04-27T07:05:00.564229+0000","last_became_peered":"2024-04-27T07:05:00.564229+0000","last_unstale":"2024-04-27T07:05:02.904096+0000","last_undegraded":"2024-04-27T07:05:02.904096+0000","last_fullsized":"2024-04-27T07:05:02.904096+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:36:50.917879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576965+0000","last_change":"2024-04-27T07:05:01.587946+0000","last_active":"2024-04-27T07:05:02.576965+0000","last_peered":"2024-04-27T07:05:02.576965+0000","last_clean":"2024-04-27T07:05:02.576965+0000","last_became_active":"2024-04-27T07:05:01.587629+0000","last_became_peered":"2024-04-27T07:05:01.587629+0000","last_unstale":"2024-04-27T07:05:02.576965+0000","last_undegraded":"2024-04-27T07:05:02.576965+0000","last_fullsized":"2024-04-27T07:05:02.576965+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:04:28.286527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.577895+0000","last_change":"2024-04-27T07:05:00.565169+0000","last_active":"2024-04-27T07:05:02.577895+0000","last_peered":"2024-04-27T07:05:02.577895+0000","last_clean":"2024-04-27T07:05:02.577895+0000","last_became_active":"2024-04-27T07:05:00.564908+0000","last_became_peered":"2024-04-27T07:05:00.564908+0000","last_unstale":"2024-04-27T07:05:02.577895+0000","last_undegraded":"2024-04-27T07:05:02.577895+0000","last_fullsized":"2024-04-27T07:05:02.577895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:32:39.772551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.575595+0000","last_change":"2024-04-27T07:05:01.578774+0000","last_active":"2024-04-27T07:05:02.575595+0000","last_peered":"2024-04-27T07:05:02.575595+0000","last_clean":"2024-04-27T07:05:02.575595+0000","last_became_active":"2024-04-27T07:05:01.578476+0000","last_became_peered":"2024-04-27T07:05:01.578476+0000","last_unstale":"2024-04-27T07:05:02.575595+0000","last_undegraded":"2024-04-27T07:05:02.575595+0000","last_fullsized":"2024-04-27T07:05:02.575595+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:32.465776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574722+0000","last_change":"2024-04-27T07:05:00.571048+0000","last_active":"2024-04-27T07:05:02.574722+0000","last_peered":"2024-04-27T07:05:02.574722+0000","last_clean":"2024-04-27T07:05:02.574722+0000","last_became_active":"2024-04-27T07:05:00.570335+0000","last_became_peered":"2024-04-27T07:05:00.570335+0000","last_unstale":"2024-04-27T07:05:02.574722+0000","last_undegraded":"2024-04-27T07:05:02.574722+0000","last_fullsized":"2024-04-27T07:05:02.574722+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:31.456531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902265+0000","last_change":"2024-04-27T07:05:01.622320+0000","last_active":"2024-04-27T07:05:02.902265+0000","last_peered":"2024-04-27T07:05:02.902265+0000","last_clean":"2024-04-27T07:05:02.902265+0000","last_became_active":"2024-04-27T07:05:01.621631+0000","last_became_peered":"2024-04-27T07:05:01.621631+0000","last_unstale":"2024-04-27T07:05:02.902265+0000","last_undegraded":"2024-04-27T07:05:02.902265+0000","last_fullsized":"2024-04-27T07:05:02.902265+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:16:21.551875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903256+0000","last_change":"2024-04-27T07:05:00.576049+0000","last_active":"2024-04-27T07:05:02.903256+0000","last_peered":"2024-04-27T07:05:02.903256+0000","last_clean":"2024-04-27T07:05:02.903256+0000","last_became_active":"2024-04-27T07:05:00.575138+0000","last_became_peered":"2024-04-27T07:05:00.575138+0000","last_unstale":"2024-04-27T07:05:02.903256+0000","last_undegraded":"2024-04-27T07:05:02.903256+0000","last_fullsized":"2024-04-27T07:05:02.903256+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:41:34.024607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576480+0000","last_change":"2024-04-27T07:05:01.615732+0000","last_active":"2024-04-27T07:05:02.576480+0000","last_peered":"2024-04-27T07:05:02.576480+0000","last_clean":"2024-04-27T07:05:02.576480+0000","last_became_active":"2024-04-27T07:05:01.615515+0000","last_became_peered":"2024-04-27T07:05:01.615515+0000","last_unstale":"2024-04-27T07:05:02.576480+0000","last_undegraded":"2024-04-27T07:05:02.576480+0000","last_fullsized":"2024-04-27T07:05:02.576480+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:03:31.976880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.906190+0000","last_change":"2024-04-27T07:05:00.579267+0000","last_active":"2024-04-27T07:05:02.906190+0000","last_peered":"2024-04-27T07:05:02.906190+0000","last_clean":"2024-04-27T07:05:02.906190+0000","last_became_active":"2024-04-27T07:05:00.578735+0000","last_became_peered":"2024-04-27T07:05:00.578735+0000","last_unstale":"2024-04-27T07:05:02.906190+0000","last_undegraded":"2024-04-27T07:05:02.906190+0000","last_fullsized":"2024-04-27T07:05:02.906190+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:32:39.772551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905648+0000","last_change":"2024-04-27T07:05:01.603236+0000","last_active":"2024-04-27T07:05:02.905648+0000","last_peered":"2024-04-27T07:05:02.905648+0000","last_clean":"2024-04-27T07:05:02.905648+0000","last_became_active":"2024-04-27T07:05:01.598615+0000","last_became_peered":"2024-04-27T07:05:01.598615+0000","last_unstale":"2024-04-27T07:05:02.905648+0000","last_undegraded":"2024-04-27T07:05:02.905648+0000","last_fullsized":"2024-04-27T07:05:02.905648+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:43:55.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.907094+0000","last_change":"2024-04-27T07:05:00.567386+0000","last_active":"2024-04-27T07:05:02.907094+0000","last_peered":"2024-04-27T07:05:02.907094+0000","last_clean":"2024-04-27T07:05:02.907094+0000","last_became_active":"2024-04-27T07:05:00.566965+0000","last_became_peered":"2024-04-27T07:05:00.566965+0000","last_unstale":"2024-04-27T07:05:02.907094+0000","last_undegraded":"2024-04-27T07:05:02.907094+0000","last_fullsized":"2024-04-27T07:05:02.907094+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:49.452830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905144+0000","last_change":"2024-04-27T07:05:01.582389+0000","last_active":"2024-04-27T07:05:02.905144+0000","last_peered":"2024-04-27T07:05:02.905144+0000","last_clean":"2024-04-27T07:05:02.905144+0000","last_became_active":"2024-04-27T07:05:01.582015+0000","last_became_peered":"2024-04-27T07:05:01.582015+0000","last_unstale":"2024-04-27T07:05:02.905144+0000","last_undegraded":"2024-04-27T07:05:02.905144+0000","last_fullsized":"2024-04-27T07:05:02.905144+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:21.852593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905687+0000","last_change":"2024-04-27T07:05:00.569614+0000","last_active":"2024-04-27T07:05:02.905687+0000","last_peered":"2024-04-27T07:05:02.905687+0000","last_clean":"2024-04-27T07:05:02.905687+0000","last_became_active":"2024-04-27T07:05:00.566282+0000","last_became_peered":"2024-04-27T07:05:00.566282+0000","last_unstale":"2024-04-27T07:05:02.905687+0000","last_undegraded":"2024-04-27T07:05:02.905687+0000","last_fullsized":"2024-04-27T07:05:02.905687+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:49.452830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.578103+0000","last_change":"2024-04-27T07:05:01.587208+0000","last_active":"2024-04-27T07:05:02.578103+0000","last_peered":"2024-04-27T07:05:02.578103+0000","last_clean":"2024-04-27T07:05:02.578103+0000","last_became_active":"2024-04-27T07:05:01.586403+0000","last_became_peered":"2024-04-27T07:05:01.586403+0000","last_unstale":"2024-04-27T07:05:02.578103+0000","last_undegraded":"2024-04-27T07:05:02.578103+0000","last_fullsized":"2024-04-27T07:05:02.578103+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:33:20.122035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.904374+0000","last_change":"2024-04-27T07:05:00.576644+0000","last_active":"2024-04-27T07:05:02.904374+0000","last_peered":"2024-04-27T07:05:02.904374+0000","last_clean":"2024-04-27T07:05:02.904374+0000","last_became_active":"2024-04-27T07:05:00.576095+0000","last_became_peered":"2024-04-27T07:05:00.576095+0000","last_unstale":"2024-04-27T07:05:02.904374+0000","last_undegraded":"2024-04-27T07:05:02.904374+0000","last_fullsized":"2024-04-27T07:05:02.904374+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:32:39.772551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903816+0000","last_change":"2024-04-27T07:05:01.613416+0000","last_active":"2024-04-27T07:05:02.903816+0000","last_peered":"2024-04-27T07:05:02.903816+0000","last_clean":"2024-04-27T07:05:02.903816+0000","last_became_active":"2024-04-27T07:05:01.613055+0000","last_became_peered":"2024-04-27T07:05:01.613055+0000","last_unstale":"2024-04-27T07:05:02.903816+0000","last_undegraded":"2024-04-27T07:05:02.903816+0000","last_fullsized":"2024-04-27T07:05:02.903816+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:33:20.122035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.905350+0000","last_change":"2024-04-27T07:05:00.577861+0000","last_active":"2024-04-27T07:05:02.905350+0000","last_peered":"2024-04-27T07:05:02.905350+0000","last_clean":"2024-04-27T07:05:02.905350+0000","last_became_active":"2024-04-27T07:05:00.577459+0000","last_became_peered":"2024-04-27T07:05:00.577459+0000","last_unstale":"2024-04-27T07:05:02.905350+0000","last_undegraded":"2024-04-27T07:05:02.905350+0000","last_fullsized":"2024-04-27T07:05:02.905350+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:22.266250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902897+0000","last_change":"2024-04-27T07:05:01.595839+0000","last_active":"2024-04-27T07:05:02.902897+0000","last_peered":"2024-04-27T07:05:02.902897+0000","last_clean":"2024-04-27T07:05:02.902897+0000","last_became_active":"2024-04-27T07:05:01.595069+0000","last_became_peered":"2024-04-27T07:05:01.595069+0000","last_unstale":"2024-04-27T07:05:02.902897+0000","last_undegraded":"2024-04-27T07:05:02.902897+0000","last_fullsized":"2024-04-27T07:05:02.902897+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T16:27:03.887951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574980+0000","last_change":"2024-04-27T07:05:00.579311+0000","last_active":"2024-04-27T07:05:02.574980+0000","last_peered":"2024-04-27T07:05:02.574980+0000","last_clean":"2024-04-27T07:05:02.574980+0000","last_became_active":"2024-04-27T07:05:00.578034+0000","last_became_peered":"2024-04-27T07:05:00.578034+0000","last_unstale":"2024-04-27T07:05:02.574980+0000","last_undegraded":"2024-04-27T07:05:02.574980+0000","last_fullsized":"2024-04-27T07:05:02.574980+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:35:41.861143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.574462+0000","last_change":"2024-04-27T07:05:01.591150+0000","last_active":"2024-04-27T07:05:02.574462+0000","last_peered":"2024-04-27T07:05:02.574462+0000","last_clean":"2024-04-27T07:05:02.574462+0000","last_became_active":"2024-04-27T07:05:01.589801+0000","last_became_peered":"2024-04-27T07:05:01.589801+0000","last_unstale":"2024-04-27T07:05:02.574462+0000","last_undegraded":"2024-04-27T07:05:02.574462+0000","last_fullsized":"2024-04-27T07:05:02.574462+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:04:17.124168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903517+0000","last_change":"2024-04-27T07:05:00.584515+0000","last_active":"2024-04-27T07:05:02.903517+0000","last_peered":"2024-04-27T07:05:02.903517+0000","last_clean":"2024-04-27T07:05:02.903517+0000","last_became_active":"2024-04-27T07:05:00.583897+0000","last_became_peered":"2024-04-27T07:05:00.583897+0000","last_unstale":"2024-04-27T07:05:02.903517+0000","last_undegraded":"2024-04-27T07:05:02.903517+0000","last_fullsized":"2024-04-27T07:05:02.903517+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:04:16.114923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.902658+0000","last_change":"2024-04-27T07:05:01.617423+0000","last_active":"2024-04-27T07:05:02.902658+0000","last_peered":"2024-04-27T07:05:02.902658+0000","last_clean":"2024-04-27T07:05:02.902658+0000","last_became_active":"2024-04-27T07:05:01.611893+0000","last_became_peered":"2024-04-27T07:05:01.611893+0000","last_unstale":"2024-04-27T07:05:02.902658+0000","last_undegraded":"2024-04-27T07:05:02.902658+0000","last_fullsized":"2024-04-27T07:05:02.902658+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:05:00.540255+0000","last_clean_scrub_stamp":"2024-04-27T07:05:00.540255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T17:22:36.544610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.576573+0000","last_change":"2024-04-27T07:05:00.572039+0000","last_active":"2024-04-27T07:05:02.576573+0000","last_peered":"2024-04-27T07:05:02.576573+0000","last_clean":"2024-04-27T07:05:02.576573+0000","last_became_active":"2024-04-27T07:05:00.571307+0000","last_became_peered":"2024-04-27T07:05:00.571307+0000","last_unstale":"2024-04-27T07:05:02.576573+0000","last_undegraded":"2024-04-27T07:05:02.576573+0000","last_fullsized":"2024-04-27T07:05:02.576573+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:59.531010+0000","last_clean_scrub_stamp":"2024-04-27T07:04:59.531010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:36:50.917879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-27T07:05:02.903264+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:05:02.903264+0000","last_peered":"2024-04-27T07:05:02.903264+0000","last_clean":"2024-04-27T07:05:02.903264+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:05:02.903264+0000","last_undegraded":"2024-04-27T07:05:02.903264+0000","last_fullsized":"2024-04-27T07:05:02.903264+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":714470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":331776,"data_stored":124628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":331776,"data_stored":124684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31508,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340332,"statfs":{"total":96636764160,"available":96604499968,"internally_reserved":0,"allocated":331776,"data_stored":124116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":327680,"data_stored":119346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":124082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":125646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":714380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:05:06.025 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-27T07:05:06.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-27T07:05:06.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b4799640 1 Processor -- start 2024-04-27T07:05:06.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b4799640 1 -- start start 2024-04-27T07:05:06.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b4799640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5ac0b58c0 0x7fa5ac0b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b4799640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa5ac141930 0x7fa5ac141da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b4799640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ac0b5120 con 0x7fa5ac0b6860 2024-04-27T07:05:06.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b4799640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fa5ac148030 con 0x7fa5ac141930 2024-04-27T07:05:06.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b4799640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ac0b5ca0 con 0x7fa5ac0b58c0 2024-04-27T07:05:06.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b1d34640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa5ac0b6860 0x7fa5ac1484c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35934/0 (socket says 172.21.15.12:35934) 2024-04-27T07:05:06.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b1d34640 1 -- 172.21.15.12:0/2168686740 learned_addr learned my addr 172.21.15.12:0/2168686740 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:06.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b2d36640 1 --2- 172.21.15.12:0/2168686740 >> v2:172.21.15.12:3300/0 conn(0x7fa5ac141930 0x7fa5ac141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b2535640 1 --2- 172.21.15.12:0/2168686740 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5ac0b58c0 0x7fa5ac0b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b1533640 1 -- 172.21.15.12:0/2168686740 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310397404 0 0) 0x7fa5ac0b5120 con 0x7fa5ac0b6860 2024-04-27T07:05:06.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b1533640 1 -- 172.21.15.12:0/2168686740 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa598003530 con 0x7fa5ac0b6860 2024-04-27T07:05:06.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b2d36640 1 -- 172.21.15.12:0/2168686740 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5ac0b58c0 msgr2=0x7fa5ac0b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:06.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b2d36640 1 --2- 172.21.15.12:0/2168686740 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5ac0b58c0 0x7fa5ac0b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.188+0000 7fa5b2d36640 1 -- 172.21.15.12:0/2168686740 >> v1:172.21.15.12:6789/0 conn(0x7fa5ac0b6860 legacy=0x7fa5ac1484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b2d36640 1 -- 172.21.15.12:0/2168686740 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac0b77f0 con 0x7fa5ac141930 2024-04-27T07:05:06.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b2d36640 1 --2- 172.21.15.12:0/2168686740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 0x7fa5ac141da0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009d90 tx=0x7fa5a401c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d35b3222c4b181 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:06.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b1533640 1 -- 172.21.15.12:0/2168686740 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40b2070 con 0x7fa5ac141930 2024-04-27T07:05:06.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b1533640 1 -- 172.21.15.12:0/2168686740 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a40a58c0 con 0x7fa5ac141930 2024-04-27T07:05:06.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b1533640 1 -- 172.21.15.12:0/2168686740 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40a5c70 con 0x7fa5ac141930 2024-04-27T07:05:06.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b4799640 1 -- 172.21.15.12:0/2168686740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 msgr2=0x7fa5ac141da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b4799640 1 --2- 172.21.15.12:0/2168686740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 0x7fa5ac141da0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009d90 tx=0x7fa5a401c820 comp rx=0 tx=0).stop 2024-04-27T07:05:06.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b4799640 1 -- 172.21.15.12:0/2168686740 shutdown_connections 2024-04-27T07:05:06.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b4799640 1 --2- 172.21.15.12:0/2168686740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 0x7fa5ac141da0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b4799640 1 --2- 172.21.15.12:0/2168686740 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5ac0b58c0 0x7fa5ac0b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b4799640 1 -- 172.21.15.12:0/2168686740 >> 172.21.15.12:0/2168686740 conn(0x7fa5ac073b90 msgr2=0x7fa5ac0bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:06.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b4799640 1 -- 172.21.15.12:0/2168686740 shutdown_connections 2024-04-27T07:05:06.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.192+0000 7fa5b4799640 1 -- 172.21.15.12:0/2168686740 wait complete. 2024-04-27T07:05:06.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 Processor -- start 2024-04-27T07:05:06.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 -- start start 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac0b58c0 0x7fa5ac1e44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5ac0b6860 0x7fa5ac1389d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 0x7fa5ac1da000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac147c50 con 0x7fa5ac141930 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac0b6f40 con 0x7fa5ac0b58c0 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ac154df0 con 0x7fa5ac0b6860 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2d36640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 0x7fa5ac1da000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2d36640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 0x7fa5ac1da000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58448/0 (socket says 172.21.15.12:58448) 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2d36640 1 -- 172.21.15.12:0/183824342 learned_addr learned my addr 172.21.15.12:0/183824342 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b1d34640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5ac0b6860 0x7fa5ac1389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2535640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac0b58c0 0x7fa5ac1e44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2d36640 1 -- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5ac0b6860 msgr2=0x7fa5ac1389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2d36640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5ac0b6860 0x7fa5ac1389d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2d36640 1 -- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac0b58c0 msgr2=0x7fa5ac1e44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2d36640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac0b58c0 0x7fa5ac1e44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2d36640 1 -- 172.21.15.12:0/183824342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac144ca0 con 0x7fa5ac141930 2024-04-27T07:05:06.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2535640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac0b58c0 0x7fa5ac1e44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:06.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b2d36640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 0x7fa5ac1da000 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40b0040 tx=0x7fa5a40195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:06.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5a2ffd640 1 -- 172.21.15.12:0/183824342 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40b2050 con 0x7fa5ac141930 2024-04-27T07:05:06.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5a2ffd640 1 -- 172.21.15.12:0/183824342 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a40be070 con 0x7fa5ac141930 2024-04-27T07:05:06.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 -- 172.21.15.12:0/183824342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac1e3a00 con 0x7fa5ac141930 2024-04-27T07:05:06.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5a2ffd640 1 -- 172.21.15.12:0/183824342 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4003db0 con 0x7fa5ac141930 2024-04-27T07:05:06.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 -- 172.21.15.12:0/183824342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac13a670 con 0x7fa5ac141930 2024-04-27T07:05:06.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5b4799640 1 -- 172.21.15.12:0/183824342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ac068c20 con 0x7fa5ac141930 2024-04-27T07:05:06.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.196+0000 7fa5a2ffd640 1 -- 172.21.15.12:0/183824342 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa5a4004040 con 0x7fa5ac141930 2024-04-27T07:05:06.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.200+0000 7fa5a2ffd640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa590068380 0x7fa59006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.200+0000 7fa5a2ffd640 1 -- 172.21.15.12:0/183824342 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fa5a4121b70 con 0x7fa5ac141930 2024-04-27T07:05:06.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.200+0000 7fa5b2535640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa590068380 0x7fa59006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.200+0000 7fa5b2535640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa590068380 0x7fa59006a840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009840 tx=0x7fa5a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:06.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.204+0000 7fa5a2ffd640 1 -- 172.21.15.12:0/183824342 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa5a4016560 con 0x7fa5ac141930 2024-04-27T07:05:06.385 INFO:tasks.ceph.mds.a.smithi012.stderr:2024-04-27T07:05:06.380+0000 7f29a960b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-27T07:05:06.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.416+0000 7fa5b4799640 1 -- 172.21.15.12:0/183824342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa5ac145670 con 0x7fa5ac141930 2024-04-27T07:05:06.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.420+0000 7fa5a2ffd640 1 -- 172.21.15.12:0/183824342 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fa5a40f2120 con 0x7fa5ac141930 2024-04-27T07:05:06.427 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-04-27T07:05:06.427 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQA+oyxmgrAhExAAceKsj8G+Q/XXODrJbKPZ0Q== 2024-04-27T07:05:06.427 INFO:teuthology.orchestra.run.smithi012.stdout: caps mds = "allow" 2024-04-27T07:05:06.427 INFO:teuthology.orchestra.run.smithi012.stdout: caps mon = "allow r" 2024-04-27T07:05:06.427 INFO:teuthology.orchestra.run.smithi012.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-27T07:05:06.427 INFO:teuthology.orchestra.run.smithi012.stderr:updated caps for client.0 2024-04-27T07:05:06.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 -- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa590068380 msgr2=0x7fa59006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa590068380 0x7fa59006a840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009840 tx=0x7fa5a8008040 comp rx=0 tx=0).stop 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 -- 172.21.15.12:0/183824342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 msgr2=0x7fa5ac1da000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 0x7fa5ac1da000 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40b0040 tx=0x7fa5a40195c0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 -- 172.21.15.12:0/183824342 shutdown_connections 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fa590068380 0x7fa59006a840 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5ac141930 0x7fa5ac1da000 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fa5ac0b6860 0x7fa5ac1389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 --2- 172.21.15.12:0/183824342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5ac0b58c0 0x7fa5ac1e44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 -- 172.21.15.12:0/183824342 >> 172.21.15.12:0/183824342 conn(0x7fa5ac073b90 msgr2=0x7fa5ac146570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 -- 172.21.15.12:0/183824342 shutdown_connections 2024-04-27T07:05:06.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.424+0000 7fa5b4799640 1 -- 172.21.15.12:0/183824342 wait complete. 2024-04-27T07:05:06.445 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:05:06.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.560+0000 7f32c8ee4640 1 Processor -- start 2024-04-27T07:05:06.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.560+0000 7f32c8ee4640 1 -- start start 2024-04-27T07:05:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c8ee4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 0x7f32c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c8ee4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f32c406d840 0x7f32c4143570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c8ee4640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32c41448e0 con 0x7f32c40b4b70 2024-04-27T07:05:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c8ee4640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f32c4144ef0 con 0x7f32c406d840 2024-04-27T07:05:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c8ee4640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f32c406dcf0 con 0x7f32c40b3db0 2024-04-27T07:05:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c2d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f32c406d840 0x7f32c4143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c2d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f32c406d840 0x7f32c4143570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58456/0 (socket says 172.21.15.12:58456) 2024-04-27T07:05:06.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c2d9d640 1 -- 172.21.15.12:0/3892551006 learned_addr learned my addr 172.21.15.12:0/3892551006 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:06.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c1d9b640 1 --2- 172.21.15.12:0/3892551006 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 0x7f32c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c2d9d640 1 -- 172.21.15.12:0/3892551006 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 msgr2=0x7f32c406d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:06.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c2d9d640 1 --2- 172.21.15.12:0/3892551006 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 0x7f32c406d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c2d9d640 1 -- 172.21.15.12:0/3892551006 >> v1:172.21.15.53:6789/0 conn(0x7f32c40b4b70 legacy=0x7f32c40b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c2d9d640 1 -- 172.21.15.12:0/3892551006 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c4144b80 con 0x7f32c406d840 2024-04-27T07:05:06.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c2d9d640 1 --2- 172.21.15.12:0/3892551006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 0x7f32c4143570 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00d150 tx=0x7f32bc01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91aa288c2df1ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:06.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c159a640 1 -- 172.21.15.12:0/3892551006 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00b2d0 con 0x7f32c406d840 2024-04-27T07:05:06.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c159a640 1 -- 172.21.15.12:0/3892551006 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32bc00b470 con 0x7f32c406d840 2024-04-27T07:05:06.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.564+0000 7f32c159a640 1 -- 172.21.15.12:0/3892551006 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc0a7c30 con 0x7f32c406d840 2024-04-27T07:05:06.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3892551006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 msgr2=0x7f32c4143570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 --2- 172.21.15.12:0/3892551006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 0x7f32c4143570 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00d150 tx=0x7f32bc01eba0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3892551006 shutdown_connections 2024-04-27T07:05:06.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 --2- 172.21.15.12:0/3892551006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 0x7f32c4143570 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 --2- 172.21.15.12:0/3892551006 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 0x7f32c406d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3892551006 >> 172.21.15.12:0/3892551006 conn(0x7f32c4067d10 msgr2=0x7f32c4068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:06.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3892551006 shutdown_connections 2024-04-27T07:05:06.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3892551006 wait complete. 2024-04-27T07:05:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 Processor -- start 2024-04-27T07:05:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- start start 2024-04-27T07:05:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 0x7f32c4142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 0x7f32c4145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32c40b4b70 0x7f32c41dc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32c4150e70 con 0x7f32c406d840 2024-04-27T07:05:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f32c4150cf0 con 0x7f32c40b4b70 2024-04-27T07:05:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f32c4150ff0 con 0x7f32c40b3db0 2024-04-27T07:05:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 0x7f32c4142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 0x7f32c4142300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58464/0 (socket says 172.21.15.12:58464) 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c259c640 1 -- 172.21.15.12:0/3067260215 learned_addr learned my addr 172.21.15.12:0/3067260215 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c1d9b640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 0x7f32c4145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c2d9d640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32c40b4b70 0x7f32c41dc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c259c640 1 -- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 msgr2=0x7f32c4145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c259c640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 0x7f32c4145390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c259c640 1 -- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32c40b4b70 msgr2=0x7f32c41dc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c259c640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32c40b4b70 0x7f32c41dc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c259c640 1 -- 172.21.15.12:0/3067260215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c414df30 con 0x7f32c406d840 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c1d9b640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 0x7f32c4145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c259c640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 0x7f32c4142300 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f32b400c370 tx=0x7f32b400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32b2ffd640 1 -- 172.21.15.12:0/3067260215 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b40052d0 con 0x7f32c406d840 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32b2ffd640 1 -- 172.21.15.12:0/3067260215 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b4005470 con 0x7f32c406d840 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3067260215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c414e210 con 0x7f32c406d840 2024-04-27T07:05:06.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32b2ffd640 1 -- 172.21.15.12:0/3067260215 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4007260 con 0x7f32c406d840 2024-04-27T07:05:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.568+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3067260215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c4142f00 con 0x7f32c406d840 2024-04-27T07:05:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.572+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3067260215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c4068c20 con 0x7f32c406d840 2024-04-27T07:05:06.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.572+0000 7f32b2ffd640 1 -- 172.21.15.12:0/3067260215 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f32b4007400 con 0x7f32c406d840 2024-04-27T07:05:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.572+0000 7f32b2ffd640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f32a4068380 0x7f32a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.572+0000 7f32b2ffd640 1 -- 172.21.15.12:0/3067260215 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f32b4085dc0 con 0x7f32c406d840 2024-04-27T07:05:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.572+0000 7f32c1d9b640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f32a4068380 0x7f32a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:06.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.572+0000 7f32c1d9b640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f32a4068380 0x7f32a406a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f32c4142b00 tx=0x7f32b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:06.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.576+0000 7f32b2ffd640 1 -- 172.21.15.12:0/3067260215 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f32b4056410 con 0x7f32c406d840 2024-04-27T07:05:06.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.776+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3067260215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32c409c930 con 0x7f32c406d840 2024-04-27T07:05:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.776+0000 7f32b2ffd640 1 -- 172.21.15.12:0/3067260215 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3531 (secure 0 0 0) 0x7f32b4055d10 con 0x7f32c406d840 2024-04-27T07:05:06.782 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:06.782 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3153112684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3153112684},{"type":"v1","addr":"172.21.15.53:6835","nonce":3153112684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:05:00.918068+0000","modified":"2024-04-27T07:05:03.947324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5001":{"gid":5001,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4067812811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4067812811},{"type":"v1","addr":"172.21.15.12:6833","nonce":4067812811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5001,"qdb_cluster":[5001]},"id":2}]} 2024-04-27T07:05:06.782 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 16 2024-04-27T07:05:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.776+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f32a4068380 msgr2=0x7f32a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.776+0000 7f32c8ee4640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f32a4068380 0x7f32a406a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f32c4142b00 tx=0x7f32b8008040 comp rx=0 tx=0).stop 2024-04-27T07:05:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.780+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3067260215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 msgr2=0x7f32c4142300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.780+0000 7f32c8ee4640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 0x7f32c4142300 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f32b400c370 tx=0x7f32b400c830 comp rx=0 tx=0).stop 2024-04-27T07:05:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.780+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3067260215 shutdown_connections 2024-04-27T07:05:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.780+0000 7f32c8ee4640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f32a4068380 0x7f32a406a840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.780+0000 7f32c8ee4640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32c40b4b70 0x7f32c41dc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.780+0000 7f32c8ee4640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f32c40b3db0 0x7f32c4145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.780+0000 7f32c8ee4640 1 --2- 172.21.15.12:0/3067260215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32c406d840 0x7f32c4142300 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.780+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3067260215 >> 172.21.15.12:0/3067260215 conn(0x7f32c4067d10 msgr2=0x7f32c40aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.780+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3067260215 shutdown_connections 2024-04-27T07:05:06.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:06.780+0000 7f32c8ee4640 1 -- 172.21.15.12:0/3067260215 wait complete. 2024-04-27T07:05:07.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-27T07:05:00.918068+0000', 'modified': '2024-04-27T07:05:03.947324+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5001}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5001': {'gid': 5001, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.12:6833/4067812811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6832', 'nonce': 4067812811}, {'type': 'v1', 'addr': '172.21.15.12:6833', 'nonce': 4067812811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5001, 'qdb_cluster': [5001]} 2024-04-27T07:05:07.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-27T07:05:07.003 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-27T07:05:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddf587f640 1 Processor -- start 2024-04-27T07:05:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddf587f640 1 -- start start 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddf587f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fddf00b6480 0x7fddf00b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddf587f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fddf01489c0 0x7fddf00b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddf587f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf00b6860 con 0x7fddf0141900 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddf587f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fddf00b84e0 con 0x7fddf01489c0 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddf587f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fddf0147ee0 con 0x7fddf00b6480 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddee7fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fddf01489c0 0x7fddf00b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddee7fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fddf01489c0 0x7fddf00b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58480/0 (socket says 172.21.15.12:58480) 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddee7fc640 1 -- 172.21.15.12:0/4066503861 learned_addr learned my addr 172.21.15.12:0/4066503861 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddeeffd640 1 --2- 172.21.15.12:0/4066503861 >> v2:172.21.15.53:3300/0 conn(0x7fddf00b6480 0x7fddf00b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddee7fc640 1 -- 172.21.15.12:0/4066503861 >> v2:172.21.15.53:3300/0 conn(0x7fddf00b6480 msgr2=0x7fddf00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddee7fc640 1 --2- 172.21.15.12:0/4066503861 >> v2:172.21.15.53:3300/0 conn(0x7fddf00b6480 0x7fddf00b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddee7fc640 1 -- 172.21.15.12:0/4066503861 >> v1:172.21.15.53:6789/0 conn(0x7fddf0141900 legacy=0x7fddf0141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.132+0000 7fddee7fc640 1 -- 172.21.15.12:0/4066503861 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf01481c0 con 0x7fddf01489c0 2024-04-27T07:05:07.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddee7fc640 1 --2- 172.21.15.12:0/4066503861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf01489c0 0x7fddf00b7fa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fddf006c2e0 tx=0x7fdde401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c6ea15a14952b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:07.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddedffb640 1 -- 172.21.15.12:0/4066503861 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde40b0070 con 0x7fddf01489c0 2024-04-27T07:05:07.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddedffb640 1 -- 172.21.15.12:0/4066503861 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde40ab600 con 0x7fddf01489c0 2024-04-27T07:05:07.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddedffb640 1 -- 172.21.15.12:0/4066503861 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde40ab9d0 con 0x7fddf01489c0 2024-04-27T07:05:07.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddf587f640 1 -- 172.21.15.12:0/4066503861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf01489c0 msgr2=0x7fddf00b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddf587f640 1 --2- 172.21.15.12:0/4066503861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf01489c0 0x7fddf00b7fa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fddf006c2e0 tx=0x7fdde401c450 comp rx=0 tx=0).stop 2024-04-27T07:05:07.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddf587f640 1 -- 172.21.15.12:0/4066503861 shutdown_connections 2024-04-27T07:05:07.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddf587f640 1 --2- 172.21.15.12:0/4066503861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf01489c0 0x7fddf00b7fa0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddf587f640 1 --2- 172.21.15.12:0/4066503861 >> v2:172.21.15.53:3300/0 conn(0x7fddf00b6480 0x7fddf00b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddf587f640 1 -- 172.21.15.12:0/4066503861 >> 172.21.15.12:0/4066503861 conn(0x7fddf00bd620 msgr2=0x7fddf00bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:07.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddf587f640 1 -- 172.21.15.12:0/4066503861 shutdown_connections 2024-04-27T07:05:07.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.136+0000 7fddf587f640 1 -- 172.21.15.12:0/4066503861 wait complete. 2024-04-27T07:05:07.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddf587f640 1 Processor -- start 2024-04-27T07:05:07.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddf587f640 1 -- start start 2024-04-27T07:05:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddf587f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf00b6480 0x7fddf013ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddf587f640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddf0141900 0x7fddf01e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddf587f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddf01489c0 0x7fddf01445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddf587f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0154ff0 con 0x7fddf00b6480 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddf587f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0154e70 con 0x7fddf01489c0 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddf587f640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fddf0155170 con 0x7fddf0141900 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddeeffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf00b6480 0x7fddf013ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddeeffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf00b6480 0x7fddf013ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58488/0 (socket says 172.21.15.12:58488) 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddeeffd640 1 -- 172.21.15.12:0/1479083420 learned_addr learned my addr 172.21.15.12:0/1479083420 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddeeffd640 1 -- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddf0141900 msgr2=0x7fddf01e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddeeffd640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddf0141900 0x7fddf01e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddeeffd640 1 -- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddf01489c0 msgr2=0x7fddf01445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddef7fe640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddf01489c0 0x7fddf01445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddeeffd640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddf01489c0 0x7fddf01445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddeeffd640 1 -- 172.21.15.12:0/1479083420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf0145400 con 0x7fddf00b6480 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddeeffd640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf00b6480 0x7fddf013ca80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d800 tx=0x7fdde000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddd7fff640 1 -- 172.21.15.12:0/1479083420 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde000b6c0 con 0x7fddf00b6480 2024-04-27T07:05:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddd7fff640 1 -- 172.21.15.12:0/1479083420 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde0019070 con 0x7fddf00b6480 2024-04-27T07:05:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddd7fff640 1 -- 172.21.15.12:0/1479083420 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde000ba60 con 0x7fddf00b6480 2024-04-27T07:05:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddf587f640 1 -- 172.21.15.12:0/1479083420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf01456a0 con 0x7fddf00b6480 2024-04-27T07:05:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddf587f640 1 -- 172.21.15.12:0/1479083420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf00bd370 con 0x7fddf00b6480 2024-04-27T07:05:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddd7fff640 1 -- 172.21.15.12:0/1479083420 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdde000bc00 con 0x7fddf00b6480 2024-04-27T07:05:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.140+0000 7fddd7fff640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdddc068380 0x7fdddc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddd7fff640 1 -- 172.21.15.12:0/1479083420 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fdde0086e20 con 0x7fddf00b6480 2024-04-27T07:05:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdddc068380 0x7fdddc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 Processor -- start 2024-04-27T07:05:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 -- start start 2024-04-27T07:05:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdddc068380 0x7fdddc06a840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdde40ad000 tx=0x7fdde4004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f86f80 0x560476f82000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f82540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 0x560476f93d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560476da6be0 con 0x560476f87360 2024-04-27T07:05:07.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560476d40bf0 con 0x560476f82a80 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x560476da2ba0 con 0x560476f86f80 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f82540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f82540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58502/0 (socket says 172.21.15.12:58502) 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 -- 172.21.15.12:0/739703850 learned_addr learned my addr 172.21.15.12:0/739703850 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddeeffd640 1 --2- 172.21.15.12:0/739703850 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f86f80 0x560476f82000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddef7fe640 1 --2- 172.21.15.12:0/739703850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 0x560476f93d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 -- 172.21.15.12:0/739703850 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f86f80 msgr2=0x560476f82000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 --2- 172.21.15.12:0/739703850 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f86f80 0x560476f82000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 -- 172.21.15.12:0/739703850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 msgr2=0x560476f93d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 --2- 172.21.15.12:0/739703850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 0x560476f93d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 -- 172.21.15.12:0/739703850 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560476e4e630 con 0x560476f87360 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddee7fc640 1 --2- 172.21.15.12:0/739703850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f82540 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdde4097d20 tx=0x7fdde40a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0136bfcd602bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddd57fa640 1 -- 172.21.15.12:0/739703850 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde40c5070 con 0x560476f87360 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddd57fa640 1 -- 172.21.15.12:0/739703850 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde40b7bb0 con 0x560476f87360 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddd57fa640 1 -- 172.21.15.12:0/739703850 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde40b0040 con 0x560476f87360 2024-04-27T07:05:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 -- 172.21.15.12:0/739703850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 msgr2=0x560476f82540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 --2- 172.21.15.12:0/739703850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f82540 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdde4097d20 tx=0x7fdde40a8000 comp rx=0 tx=0).stop 2024-04-27T07:05:07.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 -- 172.21.15.12:0/739703850 shutdown_connections 2024-04-27T07:05:07.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 --2- 172.21.15.12:0/739703850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 0x560476f93d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 --2- 172.21.15.12:0/739703850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f82540 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 --2- 172.21.15.12:0/739703850 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f86f80 0x560476f82000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 -- 172.21.15.12:0/739703850 >> 172.21.15.12:0/739703850 conn(0x560476f4fa70 msgr2=0x560476f86870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:07.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 -- 172.21.15.12:0/739703850 shutdown_connections 2024-04-27T07:05:07.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 -- 172.21.15.12:0/739703850 wait complete. 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 -- start start 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.144+0000 7fddf8a00000 1 Processor -- start 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 0x560476f5a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f86f80 0x560476f5a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f87360 0x560476f5ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560476e62d10 con 0x560476f86f80 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560476d503a0 con 0x560476f82a80 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x560476d6a1c0 con 0x560476f87360 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 10 client.0 _ll_register_callbacks cb 0x560476f43730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddee7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f86f80 0x560476f5a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddee7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f86f80 0x560476f5a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58504/0 (socket says 172.21.15.12:58504) 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddee7fc640 1 -- 172.21.15.12:0/1293023818 learned_addr learned my addr 172.21.15.12:0/1293023818 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddef7fe640 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f87360 0x560476f5ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddeeffd640 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 0x560476f5a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddee7fc640 1 -- 172.21.15.12:0/1293023818 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f87360 msgr2=0x560476f5ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddee7fc640 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f87360 0x560476f5ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddee7fc640 1 -- 172.21.15.12:0/1293023818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 msgr2=0x560476f5a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddee7fc640 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 0x560476f5a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddee7fc640 1 -- 172.21.15.12:0/1293023818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560476dec0d0 con 0x560476f86f80 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddef7fe640 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f87360 0x560476f5ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddeeffd640 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 0x560476f5a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddee7fc640 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f86f80 0x560476f5a8e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdde4006db0 tx=0x7fdde401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c968bf7cd24bcaa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddce7fc640 1 -- 172.21.15.12:0/1293023818 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde40c5040 con 0x560476f86f80 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddce7fc640 1 -- 172.21.15.12:0/1293023818 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fdde4004230 con 0x560476f86f80 2024-04-27T07:05:07.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- 172.21.15.12:0/1293023818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde40b7bb0 con 0x560476f86f80 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddce7fc640 1 -- 172.21.15.12:0/1293023818 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fdde4004430 con 0x560476f86f80 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 10 client.5166 fetch_fsmap learned FSMap version 16 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- 172.21.15.12:0/1293023818 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x560476d2ce80 con 0x560476f86f80 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddce7fc640 1 -- 172.21.15.12:0/1293023818 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde40b7bb0 con 0x560476f86f80 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddce7fc640 1 -- 172.21.15.12:0/1293023818 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 16) ==== 2284+0+0 (secure 0 0 0) 0x7fdde40bf070 con 0x560476f86f80 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 10 client.5166 fetch_fsmap finished waiting for FSMap version 16 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 10 client.5166 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 20 client.5166 populate_metadata read hostname 'smithi012' 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.12:6832/4067812811,v1:172.21.15.12:6833/4067812811] conn(0x560476f9d0f0 0x560476f9f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 4 client.5166 mds_command: new command op to 5001 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- 172.21.15.12:0/1293023818 --> [v2:172.21.15.12:6832/4067812811,v1:172.21.15.12:6833/4067812811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560476d3fbc0 con 0x560476f9d0f0 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddeeffd640 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.12:6832/4067812811,v1:172.21.15.12:6833/4067812811] conn(0x560476f9d0f0 0x560476f9f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddeeffd640 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.12:6832/4067812811,v1:172.21.15.12:6833/4067812811] conn(0x560476f9d0f0 0x560476f9f4e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c5cd5ba545910b server_cookie=b8caaf9b22762031 in_seq=0 out_seq=0 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddce7fc640 10 client.5166 ms_handle_connect on v2:172.21.15.12:6832/4067812811 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddce7fc640 1 -- 172.21.15.12:0/1293023818 <== mds.0 v2:172.21.15.12:6832/4067812811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560476d3fbc0 con 0x560476f9d0f0 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddce7fc640 10 client.5166 handle_command_reply: tid=0 2024-04-27T07:05:07.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 client.5166 shutdown 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- 172.21.15.12:0/1293023818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f86f80 msgr2=0x560476f5a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f86f80 0x560476f5a8e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdde4006db0 tx=0x7fdde401ac90 comp rx=0 tx=0).stop 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- 172.21.15.12:0/1293023818 shutdown_connections 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f87360 0x560476f5ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.12:6832/4067812811,v1:172.21.15.12:6833/4067812811] conn(0x560476f9d0f0 0x560476f9f4e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f86f80 0x560476f5a8e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 --2- 172.21.15.12:0/1293023818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f82a80 0x560476f5a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddce7fc640 0 client.5166 ms_handle_reset on v2:172.21.15.12:6832/4067812811 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- 172.21.15.12:0/1293023818 >> 172.21.15.12:0/1293023818 conn(0x560476f4fa70 msgr2=0x560476f86870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- 172.21.15.12:0/1293023818 shutdown_connections 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 1 -- 172.21.15.12:0/1293023818 wait complete. 2024-04-27T07:05:07.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.148+0000 7fddf8a00000 20 client.5166 trim_cache size 0 max 16384 2024-04-27T07:05:07.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 Processor -- start 2024-04-27T07:05:07.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 -- start start 2024-04-27T07:05:07.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f82a80 0x560476f5de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f86f80 0x560476f58b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f59060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fddf01ecfe0 con 0x560476f87360 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fddf01ec270 con 0x560476f86f80 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x560476db1d00 con 0x560476f82a80 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddef7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f59060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddef7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f59060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58516/0 (socket says 172.21.15.12:58516) 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddef7fe640 1 -- 172.21.15.12:0/727838126 learned_addr learned my addr 172.21.15.12:0/727838126 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddef7fe640 1 -- 172.21.15.12:0/727838126 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f82a80 msgr2=0x560476f5de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddef7fe640 1 --2- 172.21.15.12:0/727838126 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f82a80 0x560476f5de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddef7fe640 1 -- 172.21.15.12:0/727838126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f86f80 msgr2=0x560476f58b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddef7fe640 1 --2- 172.21.15.12:0/727838126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f86f80 0x560476f58b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddef7fe640 1 -- 172.21.15.12:0/727838126 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560476d6eb30 con 0x560476f87360 2024-04-27T07:05:07.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddef7fe640 1 --2- 172.21.15.12:0/727838126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f59060 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdde80053e0 tx=0x7fdde80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8a1fa138203ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:07.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddd4ff9640 1 -- 172.21.15.12:0/727838126 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8017020 con 0x560476f87360 2024-04-27T07:05:07.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddd4ff9640 1 -- 172.21.15.12:0/727838126 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde800fcc0 con 0x560476f87360 2024-04-27T07:05:07.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddd4ff9640 1 -- 172.21.15.12:0/727838126 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8013670 con 0x560476f87360 2024-04-27T07:05:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 -- 172.21.15.12:0/727838126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 msgr2=0x560476f59060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 --2- 172.21.15.12:0/727838126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f59060 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdde80053e0 tx=0x7fdde80058a0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 -- 172.21.15.12:0/727838126 shutdown_connections 2024-04-27T07:05:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 --2- 172.21.15.12:0/727838126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f87360 0x560476f59060 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 --2- 172.21.15.12:0/727838126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f86f80 0x560476f58b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 --2- 172.21.15.12:0/727838126 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f82a80 0x560476f5de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 -- 172.21.15.12:0/727838126 >> 172.21.15.12:0/727838126 conn(0x560476f4fa70 msgr2=0x560476f5d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:07.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 -- 172.21.15.12:0/727838126 shutdown_connections 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 -- 172.21.15.12:0/727838126 wait complete. 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 -- start start 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.168+0000 7fddf8a00000 1 Processor -- start 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f82a80 0x560476fa0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f86f80 0x560476fa0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f87360 0x560476fa0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560476dab210 con 0x560476f82a80 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560476da5ef0 con 0x560476f87360 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x560476db3fe0 con 0x560476f86f80 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 10 client.0 _ll_register_callbacks cb 0x7fdde40c3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddeeffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f82a80 0x560476fa0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddeeffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f82a80 0x560476fa0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58522/0 (socket says 172.21.15.12:58522) 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddeeffd640 1 -- 172.21.15.12:0/2306330065 learned_addr learned my addr 172.21.15.12:0/2306330065 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddee7fc640 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f86f80 0x560476fa0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddeeffd640 1 -- 172.21.15.12:0/2306330065 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f86f80 msgr2=0x560476fa0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:07.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddeeffd640 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f86f80 0x560476fa0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddef7fe640 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f87360 0x560476fa0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddeeffd640 1 -- 172.21.15.12:0/2306330065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f87360 msgr2=0x560476fa0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddeeffd640 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f87360 0x560476fa0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddeeffd640 1 -- 172.21.15.12:0/2306330065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560476e27a10 con 0x560476f82a80 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddef7fe640 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f87360 0x560476fa0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddeeffd640 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f82a80 0x560476fa0530 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdde0056d00 tx=0x7fdde0053010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc879476603597b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddce7fc640 1 -- 172.21.15.12:0/2306330065 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde00533e0 con 0x560476f82a80 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddce7fc640 1 -- 172.21.15.12:0/2306330065 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fdde006f130 con 0x560476f82a80 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- 172.21.15.12:0/2306330065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdde800fcc0 con 0x560476f82a80 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddce7fc640 1 -- 172.21.15.12:0/2306330065 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fdde0053580 con 0x560476f82a80 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 10 client.5172 fetch_fsmap learned FSMap version 16 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- 172.21.15.12:0/2306330065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x560476dfee30 con 0x560476f82a80 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddce7fc640 1 -- 172.21.15.12:0/2306330065 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde800fcc0 con 0x560476f82a80 2024-04-27T07:05:07.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddce7fc640 1 -- 172.21.15.12:0/2306330065 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 16) ==== 2284+0+0 (secure 0 0 0) 0x7fdde0013070 con 0x560476f82a80 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 10 client.5172 fetch_fsmap finished waiting for FSMap version 16 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 10 client.5172 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 20 client.5172 populate_metadata read hostname 'smithi012' 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.12:6832/4067812811,v1:172.21.15.12:6833/4067812811] conn(0x560477003430 0x560477005820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 4 client.5172 mds_command: new command op to 5001 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- 172.21.15.12:0/2306330065 --> [v2:172.21.15.12:6832/4067812811,v1:172.21.15.12:6833/4067812811] -- command(tid 0: {"prefix": "status"}) -- 0x560476db47f0 con 0x560477003430 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddee7fc640 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.12:6832/4067812811,v1:172.21.15.12:6833/4067812811] conn(0x560477003430 0x560477005820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddee7fc640 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.12:6832/4067812811,v1:172.21.15.12:6833/4067812811] conn(0x560477003430 0x560477005820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e50a72488f5dc6 server_cookie=3873b73b883726e7 in_seq=0 out_seq=0 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddce7fc640 10 client.5172 ms_handle_connect on v2:172.21.15.12:6832/4067812811 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddce7fc640 1 -- 172.21.15.12:0/2306330065 <== mds.0 v2:172.21.15.12:6832/4067812811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x560476db47f0 con 0x560477003430 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddce7fc640 10 client.5172 handle_command_reply: tid=0 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 client.5172 shutdown 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- 172.21.15.12:0/2306330065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f82a80 msgr2=0x560476fa0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f82a80 0x560476fa0530 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdde0056d00 tx=0x7fdde0053010 comp rx=0 tx=0).stop 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- 172.21.15.12:0/2306330065 shutdown_connections 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560476f87360 0x560476fa0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x560476f86f80 0x560476fa0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.12:6832/4067812811,v1:172.21.15.12:6833/4067812811] conn(0x560477003430 0x560477005820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 --2- 172.21.15.12:0/2306330065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560476f82a80 0x560476fa0530 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddce7fc640 0 client.5172 ms_handle_reset on v2:172.21.15.12:6832/4067812811 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- 172.21.15.12:0/2306330065 >> 172.21.15.12:0/2306330065 conn(0x560476f4fa70 msgr2=0x560476fa2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- 172.21.15.12:0/2306330065 shutdown_connections 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 1 -- 172.21.15.12:0/2306330065 wait complete. 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf8a00000 20 client.5172 trim_cache size 0 max 16384 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "e023690c-eecf-4ce8-af7e-ab331203966f", 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-04-27T07:05:07.184 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 5001, 2024-04-27T07:05:07.185 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-27T07:05:07.185 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-27T07:05:07.185 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-27T07:05:07.185 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 5.6103230120000003, 2024-04-27T07:05:07.185 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 16, 2024-04-27T07:05:07.185 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 29, 2024-04-27T07:05:07.185 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 29, 2024-04-27T07:05:07.185 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 12.677400786 2024-04-27T07:05:07.185 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 -- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdddc068380 msgr2=0x7fdddc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdddc068380 0x7fdddc06a840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdde40ad000 tx=0x7fdde4004740 comp rx=0 tx=0).stop 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 -- 172.21.15.12:0/1479083420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf00b6480 msgr2=0x7fddf013ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf00b6480 0x7fddf013ca80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d800 tx=0x7fdde000dcc0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 -- 172.21.15.12:0/1479083420 shutdown_connections 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdddc068380 0x7fdddc06a840 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fddf01489c0 0x7fddf01445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fddf0141900 0x7fddf01e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 --2- 172.21.15.12:0/1479083420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf00b6480 0x7fddf013ca80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 -- 172.21.15.12:0/1479083420 >> 172.21.15.12:0/1479083420 conn(0x7fddf00bd620 msgr2=0x7fddf0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 -- 172.21.15.12:0/1479083420 shutdown_connections 2024-04-27T07:05:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.172+0000 7fddf587f640 1 -- 172.21.15.12:0/1479083420 wait complete. 2024-04-27T07:05:07.195 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-27T07:05:07.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.304+0000 7ff901812640 1 Processor -- start 2024-04-27T07:05:07.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.304+0000 7ff901812640 1 -- start start 2024-04-27T07:05:07.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff8fc0b9ae0 0x7ff8fc073c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8fc0b9ec0 con 0x7ff8fc13e5b0 2024-04-27T07:05:07.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8fc09ddb0 con 0x7ff8fc1480a0 2024-04-27T07:05:07.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7ff8fc09e090 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8faffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff8fc0b9ae0 0x7ff8fc073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8fb7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff8fc13e5b0 0x7ff8fc13ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35940/0 (socket says 172.21.15.12:35940) 2024-04-27T07:05:07.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8fb7fe640 1 -- 172.21.15.12:0/4048698854 learned_addr learned my addr 172.21.15.12:0/4048698854 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:07.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8faffd640 1 -- 172.21.15.12:0/4048698854 >> v1:172.21.15.53:6789/0 conn(0x7ff8fc1480a0 legacy=0x7ff8fc148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8faffd640 1 -- 172.21.15.12:0/4048698854 >> v1:172.21.15.12:6789/0 conn(0x7ff8fc13e5b0 legacy=0x7ff8fc13ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8f9ffb640 1 -- 172.21.15.12:0/4048698854 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1257526038 0 0) 0x7ff8fc0b9ec0 con 0x7ff8fc13e5b0 2024-04-27T07:05:07.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8faffd640 1 -- 172.21.15.12:0/4048698854 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc09d980 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8faffd640 1 --2- 172.21.15.12:0/4048698854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 0x7ff8fc073c60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009dc0 tx=0x7ff8f401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d8546891bc3fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8f9ffb640 1 -- 172.21.15.12:0/4048698854 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40b2070 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8f9ffb640 1 -- 172.21.15.12:0/4048698854 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f40a5810 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff8f9ffb640 1 -- 172.21.15.12:0/4048698854 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40a5be0 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 -- 172.21.15.12:0/4048698854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 msgr2=0x7ff8fc073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 --2- 172.21.15.12:0/4048698854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 0x7ff8fc073c60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009dc0 tx=0x7ff8f401c950 comp rx=0 tx=0).stop 2024-04-27T07:05:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 -- 172.21.15.12:0/4048698854 shutdown_connections 2024-04-27T07:05:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 --2- 172.21.15.12:0/4048698854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 0x7ff8fc073c60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 -- 172.21.15.12:0/4048698854 >> 172.21.15.12:0/4048698854 conn(0x7ff8fc067d10 msgr2=0x7ff8fc068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 -- 172.21.15.12:0/4048698854 shutdown_connections 2024-04-27T07:05:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.308+0000 7ff901812640 1 -- 172.21.15.12:0/4048698854 wait complete. 2024-04-27T07:05:07.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 Processor -- start 2024-04-27T07:05:07.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 -- start start 2024-04-27T07:05:07.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 0x7ff8fc0b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8fc13e5b0 0x7ff8fc1e1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc1480a0 0x7ff8fc1374d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc152cc0 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc0b5110 con 0x7ff8fc1480a0 2024-04-27T07:05:07.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7ff8fc152e40 con 0x7ff8fc13e5b0 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8faffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 0x7ff8fc0b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8faffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 0x7ff8fc0b8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58542/0 (socket says 172.21.15.12:58542) 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8faffd640 1 -- 172.21.15.12:0/638355740 learned_addr learned my addr 172.21.15.12:0/638355740 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8fb7fe640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc1480a0 0x7ff8fc1374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8fa7fc640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8fc13e5b0 0x7ff8fc1e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8faffd640 1 -- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8fc13e5b0 msgr2=0x7ff8fc1e1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8faffd640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8fc13e5b0 0x7ff8fc1e1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8faffd640 1 -- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc1480a0 msgr2=0x7ff8fc1374d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8faffd640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc1480a0 0x7ff8fc1374d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8faffd640 1 -- 172.21.15.12:0/638355740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc0b8c00 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8fb7fe640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc1480a0 0x7ff8fc1374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8faffd640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 0x7ff8fc0b8460 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009dc0 tx=0x7ff8f40196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8e3fff640 1 -- 172.21.15.12:0/638355740 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40bd030 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8e3fff640 1 -- 172.21.15.12:0/638355740 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f40a5610 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 -- 172.21.15.12:0/638355740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc13b960 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8e3fff640 1 -- 172.21.15.12:0/638355740 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40b2040 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 -- 172.21.15.12:0/638355740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc1386a0 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff901812640 1 -- 172.21.15.12:0/638355740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8fc068c20 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.312+0000 7ff8e3fff640 1 -- 172.21.15.12:0/638355740 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff8f40c0070 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.316+0000 7ff8e3fff640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff8e8068380 0x7ff8e806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:07.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.316+0000 7ff8e3fff640 1 -- 172.21.15.12:0/638355740 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7ff8f4121780 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.316+0000 7ff8fa7fc640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff8e8068380 0x7ff8e806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:07.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.316+0000 7ff8fa7fc640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff8e8068380 0x7ff8e806a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc1e2210 tx=0x7ff8f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:07.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.320+0000 7ff8e3fff640 1 -- 172.21.15.12:0/638355740 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff8f4016540 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.516+0000 7ff901812640 1 -- 172.21.15.12:0/638355740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff8fc142320 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.516+0000 7ff8e3fff640 1 -- 172.21.15.12:0/638355740 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7ff8f40f1ef0 con 0x7ff8fc0b9ae0 2024-04-27T07:05:07.523 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 -- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff8e8068380 msgr2=0x7ff8e806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff8e8068380 0x7ff8e806a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc1e2210 tx=0x7ff8f0009040 comp rx=0 tx=0).stop 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 -- 172.21.15.12:0/638355740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 msgr2=0x7ff8fc0b8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 0x7ff8fc0b8460 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009dc0 tx=0x7ff8f40196f0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 -- 172.21.15.12:0/638355740 shutdown_connections 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff8e8068380 0x7ff8e806a840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8fc1480a0 0x7ff8fc1374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff8fc13e5b0 0x7ff8fc1e1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 --2- 172.21.15.12:0/638355740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8fc0b9ae0 0x7ff8fc0b8460 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 -- 172.21.15.12:0/638355740 >> 172.21.15.12:0/638355740 conn(0x7ff8fc067d10 msgr2=0x7ff8fc09b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 -- 172.21.15.12:0/638355740 shutdown_connections 2024-04-27T07:05:07.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07.520+0000 7ff901812640 1 -- 172.21.15.12:0/638355740 wait complete. 2024-04-27T07:05:07.540 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-27T07:05:07.540 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-27T07:05:07.540 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-04-27T07:05:07.540 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-27T07:05:07.540 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-27T07:05:07.540 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-27T07:05:07.540 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-27T07:05:07.540 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-27T07:05:07.541 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-27T07:05:07.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:07.541 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-04-27T07:05:07.547 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42435sec preferred_lft 42435sec 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 02:42:2a:2f:06:4a brd ff:ff:ff:ff:ff:ff 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-27T07:05:07.548 INFO:teuthology.orchestra.run.smithi012.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-27T07:05:07.549 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:05:07.549 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-27T07:05:07.549 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:05:07.549 INFO:teuthology.orchestra.run.smithi012.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-27T07:05:07.549 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:05:07.549 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-27T07:05:07.549 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-27T07:05:07.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:07.549 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-27T07:05:07.600 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-04-27T07:05:07.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:07.600 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-04-27T07:05:07.651 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-04-27T07:05:07.651 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-27T07:05:07.651 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-27T07:05:07.651 INFO:teuthology.orchestra.run.smithi012.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-27T07:05:07.651 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-27T07:05:07.651 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-27T07:05:07.651 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-27T07:05:07.651 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-27T07:05:07.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-27T07:05:07.652 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-27T07:05:07.652 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-27T07:05:07.652 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-27T07:05:07.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-27T07:05:07.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:05:07.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:07.773 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-27T07:05:07.779 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-27T07:05:07.779 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-27T07:05:07.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:07.779 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-04-27T07:05:07.833 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-27T07:05:07.833 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:05:07.833 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-27T07:05:07.881 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-27T07:05:07.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-27T07:05:07.881 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-27T07:05:07.928 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-04-27T07:05:07.986 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:05:08.034 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-04-27T07:05:08.034 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-04-27T07:05:08.034 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-04-27T07:05:08.034 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-04-27T07:05:08.034 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-04-27T07:05:08.034 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-04-27T07:05:08.034 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-04-27T07:05:08.034 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/run/lock 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/fs/nfsd 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-04-27T07:05:08.035 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/run/rpc_pipefs 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:05:08.036 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:05:08.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:08.037 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-04-27T07:05:08.085 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-27T07:05:08.085 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-27T07:05:08.130 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-04-27T07:05:08.145 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:05:08.193 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-04-27T07:05:08.193 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-04-27T07:05:08.193 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-04-27T07:05:08.193 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-04-27T07:05:08.193 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-04-27T07:05:08.193 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-04-27T07:05:08.193 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/run/lock 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/fs/nfsd 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:05:08.194 INFO:teuthology.orchestra.run.smithi012.stdout:/run/rpc_pipefs 2024-04-27T07:05:08.195 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-04-27T07:05:08.195 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-04-27T07:05:08.195 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:05:08.195 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:05:08.195 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:05:08.196 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:05:08.196 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-04-27T07:05:08.196 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:05:08.196 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:05:08.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:08.197 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-04-27T07:05:08.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:2024-04-27T07:05:08.204+0000 7f239a27e580 -1 init, newargv = 0x55fac2526ba0 newargc=15 2024-04-27T07:05:08.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:ceph-fuse[36912]: starting ceph client 2024-04-27T07:05:08.214 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:ceph-fuse[36912]: starting fuse 2024-04-27T07:05:09.314 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-04-27T07:05:09.331 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-04-27T07:05:09.378 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-04-27T07:05:09.378 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-04-27T07:05:09.378 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-04-27T07:05:09.378 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/run/lock 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/fs/nfsd 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-04-27T07:05:09.379 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysusers.service 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/run/rpc_pipefs 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-0 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-1 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-2 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-3 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:05:09.380 INFO:teuthology.orchestra.run.smithi012.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-27T07:05:09.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:09.381 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-04-27T07:05:09.433 INFO:teuthology.orchestra.run.smithi012.stdout:45 2024-04-27T07:05:09.434 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo stdin-killer -- python3 -c ' 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> import glob 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> import re 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> import os 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> import subprocess 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> def _find_admin_socket(client_name): 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> files = glob.glob(asok_path) 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> # Given a non-glob path, it better be there 2024-04-27T07:05:09.434 DEBUG:teuthology.orchestra.run.smithi012:> if "*" not in asok_path: 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> assert(len(files) == 1) 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> return files[0] 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> for f in files: 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> contents = proc_f.read() 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> if mountpoint in contents: 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> return f 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> print(_find_admin_socket("client.0")) 2024-04-27T07:05:09.435 DEBUG:teuthology.orchestra.run.smithi012:> ' 2024-04-27T07:05:09.561 INFO:teuthology.orchestra.run.smithi012.stdout:/var/run/ceph/ceph-client.0.36912.asok 2024-04-27T07:05:09.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:05:09.573 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36912.asok 2024-04-27T07:05:09.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:09.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36912.asok status 2024-04-27T07:05:09.729 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:05:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "metadata": { 2024-04-27T07:05:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_sha1": "c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f", 2024-04-27T07:05:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-3227-gc37f63b3 (c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f) squid (dev)", 2024-04-27T07:05:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "entity_id": "0", 2024-04-27T07:05:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": "smithi012", 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "pid": "36912", 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "root": "/" 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_count": 0, 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_pinned_count": 0, 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 5184, 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "inst": { 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": { 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "client", 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "num": 5184 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-04-27T07:05:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2035887244 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2035887244 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "inst_str": "client.5184 192.168.0.1:0/2035887244", 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "addr_str": "192.168.0.1:0/2035887244", 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "inode_count": 1, 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_epoch": 16, 2024-04-27T07:05:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch": 29, 2024-04-27T07:05:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch_barrier": 0, 2024-04-27T07:05:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklisted": false, 2024-04-27T07:05:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs" 2024-04-27T07:05:09.732 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:05:09.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:09.743 DEBUG:teuthology.orchestra.run.smithi012:> stat --file-system '--printf=%T 2024-04-27T07:05:09.744 DEBUG:teuthology.orchestra.run.smithi012:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-27T07:05:09.749 INFO:teuthology.orchestra.run.smithi012.stdout:fuseblk 2024-04-27T07:05:09.749 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-27T07:05:09.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:09.750 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-27T07:05:09.809 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-04-27T07:05:09.857 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.2GpJ7fLi7L 2024-04-27T07:05:09.857 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:05:09.857 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/tmp/tmp.2GpJ7fLi7L 2024-04-27T07:05:09.906 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p dir1/dir2/dir3') 2024-04-27T07:05:09.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-27T07:05:10.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:05:10.038 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 755 /tmp/test-cephfsxv5ouxx1') 2024-04-27T07:05:10.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-27T07:05:10.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-27T07:05:10.097 INFO:tasks.cephfs.test_cephfs_shell:Running script "/tmp/test-cephfsxv5ouxx1" 2024-04-27T07:05:10.097 DEBUG:teuthology.orchestra.run.smithi012:> cephfs-shell -b /tmp/test-cephfsxv5ouxx1 2024-04-27T07:05:10.377 INFO:teuthology.orchestra.run.smithi012.stderr:Unrecognized command 2024-04-27T07:05:10.405 DEBUG:teuthology.orchestra.run:got remote process result: 127 2024-04-27T07:05:10.406 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-27T07:05:10.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-27T07:05:10.470 DEBUG:teuthology.orchestra.run.smithi053:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-27T07:05:10.516 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-27T07:05:10.516 INFO:tasks.cephfs.mount:Terminating background process 2024-04-27T07:05:10.516 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:05:10.516 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:05:10.522 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:05:10.522 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:05:10.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:05:10.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6f7fb640 1 Processor -- start 2024-04-27T07:05:10.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6f7fb640 1 -- start start 2024-04-27T07:05:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6f7fb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fef680b9a10 0x7fef680b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6f7fb640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef680b3680 0x7fef68145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6f7fb640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef6806dac0 con 0x7fef681460b0 2024-04-27T07:05:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6f7fb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fef6806e890 con 0x7fef680b9a10 2024-04-27T07:05:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6f7fb640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fef680b9df0 con 0x7fef680b3680 2024-04-27T07:05:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6dd98640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fef681460b0 0x7fef681490e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56242/0 (socket says 172.21.15.12:56242) 2024-04-27T07:05:10.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6dd98640 1 -- 172.21.15.12:0/237015734 learned_addr learned my addr 172.21.15.12:0/237015734 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6d597640 1 --2- 172.21.15.12:0/237015734 >> v2:172.21.15.53:3300/0 conn(0x7fef680b9a10 0x7fef680b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 235758293 0 0) 0x7fef600014a0 con 0x7fef681460b0 2024-04-27T07:05:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6cd96640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef680b3680 0x7fef68145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef54003530 con 0x7fef681460b0 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1272476317 0 0) 0x7fef54003530 con 0x7fef681460b0 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef680b3680 msgr2=0x7fef68145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef680b3680 0x7fef68145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 >> v2:172.21.15.53:3300/0 conn(0x7fef680b9a10 msgr2=0x7fef680b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 --2- 172.21.15.12:0/237015734 >> v2:172.21.15.53:3300/0 conn(0x7fef680b9a10 0x7fef680b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6806e090 con 0x7fef681460b0 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fef60001bf0 con 0x7fef681460b0 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 >> v1:172.21.15.12:6789/0 conn(0x7fef681460b0 legacy=0x7fef681490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef540039c0 0x7fef54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef54004810 0x7fef54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef680b3680 0x7fef540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef6806e890 con 0x7fef54004810 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef680b9df0 con 0x7fef540039c0 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fef5400b9d0 con 0x7fef680b3680 2024-04-27T07:05:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6cd96640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef54004810 0x7fef54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6d597640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef540039c0 0x7fef54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6dd98640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef680b3680 0x7fef540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6cd96640 1 -- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef680b3680 msgr2=0x7fef540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6cd96640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef680b3680 0x7fef540051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6cd96640 1 -- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef540039c0 msgr2=0x7fef54003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6cd96640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef540039c0 0x7fef54003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.740+0000 7fef6cd96640 1 -- 172.21.15.12:0/237015734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef54003530 con 0x7fef54004810 2024-04-27T07:05:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6cd96640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef54004810 0x7fef54004c90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fef64009af0 tx=0x7fef6401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2fe360a532cf18 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64009690 con 0x7fef54004810 2024-04-27T07:05:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef640a9430 con 0x7fef54004810 2024-04-27T07:05:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef5ffff640 1 -- 172.21.15.12:0/237015734 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef640a9800 con 0x7fef54004810 2024-04-27T07:05:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 -- 172.21.15.12:0/237015734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef54004810 msgr2=0x7fef54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 --2- 172.21.15.12:0/237015734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef54004810 0x7fef54004c90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fef64009af0 tx=0x7fef6401ca30 comp rx=0 tx=0).stop 2024-04-27T07:05:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6dd98640 1 -- 172.21.15.12:0/237015734 reap_dead start 2024-04-27T07:05:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 -- 172.21.15.12:0/237015734 shutdown_connections 2024-04-27T07:05:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 -- 172.21.15.12:0/237015734 >> 172.21.15.12:0/237015734 conn(0x7fef68067d10 msgr2=0x7fef68068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 -- 172.21.15.12:0/237015734 shutdown_connections 2024-04-27T07:05:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 -- 172.21.15.12:0/237015734 wait complete. 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 Processor -- start 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 -- start start 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef54004810 0x7fef681481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6809e070 0x7fef6814a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef681dede0 0x7fef681e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef680b6b00 con 0x7fef6809e070 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef680740b0 con 0x7fef681dede0 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6f7fb640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fef680b6c80 con 0x7fef54004810 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6cd96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6809e070 0x7fef6814a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6cd96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6809e070 0x7fef6814a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35070/0 (socket says 172.21.15.12:35070) 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6cd96640 1 -- 172.21.15.12:0/1643093506 learned_addr learned my addr 172.21.15.12:0/1643093506 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6d597640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef54004810 0x7fef681481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6cd96640 1 -- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef54004810 msgr2=0x7fef681481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6dd98640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef681dede0 0x7fef681e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6cd96640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef54004810 0x7fef681481d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6cd96640 1 -- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef681dede0 msgr2=0x7fef681e23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6cd96640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef681dede0 0x7fef681e23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6cd96640 1 -- 172.21.15.12:0/1643093506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6814a870 con 0x7fef6809e070 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.744+0000 7fef6cd96640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6809e070 0x7fef6814a330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fef6401c860 tx=0x7fef640197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef5dffb640 1 -- 172.21.15.12:0/1643093506 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef640a9ce0 con 0x7fef6809e070 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef5dffb640 1 -- 172.21.15.12:0/1643093506 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef640bc430 con 0x7fef6809e070 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef6f7fb640 1 -- 172.21.15.12:0/1643093506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6814b0c0 con 0x7fef6809e070 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef5dffb640 1 -- 172.21.15.12:0/1643093506 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef640bc820 con 0x7fef6809e070 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef6f7fb640 1 -- 172.21.15.12:0/1643093506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef68070900 con 0x7fef6809e070 2024-04-27T07:05:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef6f7fb640 1 -- 172.21.15.12:0/1643093506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef68068c20 con 0x7fef6809e070 2024-04-27T07:05:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef5dffb640 1 -- 172.21.15.12:0/1643093506 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fef640c3060 con 0x7fef6809e070 2024-04-27T07:05:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef5dffb640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef4c068380 0x7fef4c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef5dffb640 1 -- 172.21.15.12:0/1643093506 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fef64120ee0 con 0x7fef6809e070 2024-04-27T07:05:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef6d597640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef4c068380 0x7fef4c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.748+0000 7fef6d597640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef4c068380 0x7fef4c06a840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fef580099b0 tx=0x7fef58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:10.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.752+0000 7fef5dffb640 1 -- 172.21.15.12:0/1643093506 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fef640ad020 con 0x7fef6809e070 2024-04-27T07:05:10.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.916+0000 7fef6f7fb640 1 -- 172.21.15.12:0/1643093506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef6806d7f0 con 0x7fef6809e070 2024-04-27T07:05:10.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.920+0000 7fef5dffb640 1 -- 172.21.15.12:0/1643093506 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16609 (secure 0 0 0) 0x7fef64016780 con 0x7fef6809e070 2024-04-27T07:05:10.929 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:10.929 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:05:02.565909+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:59.279361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:59.893681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:05:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 -- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef4c068380 msgr2=0x7fef4c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef4c068380 0x7fef4c06a840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fef580099b0 tx=0x7fef58008040 comp rx=0 tx=0).stop 2024-04-27T07:05:10.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 -- 172.21.15.12:0/1643093506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6809e070 msgr2=0x7fef6814a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6809e070 0x7fef6814a330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fef6401c860 tx=0x7fef640197d0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 -- 172.21.15.12:0/1643093506 shutdown_connections 2024-04-27T07:05:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fef4c068380 0x7fef4c06a840 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef681dede0 0x7fef681e23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef6809e070 0x7fef6814a330 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 --2- 172.21.15.12:0/1643093506 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fef54004810 0x7fef681481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 -- 172.21.15.12:0/1643093506 >> 172.21.15.12:0/1643093506 conn(0x7fef68067d10 msgr2=0x7fef680b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 -- 172.21.15.12:0/1643093506 shutdown_connections 2024-04-27T07:05:10.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:10.924+0000 7fef6f7fb640 1 -- 172.21.15.12:0/1643093506 wait complete. 2024-04-27T07:05:10.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:59.279361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:59.893681+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:05:10.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:05:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9463e8d640 1 Processor -- start 2024-04-27T07:05:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9463e8d640 1 -- start start 2024-04-27T07:05:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9463e8d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f945c0b4ae0 0x7f945c1454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9463e8d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f945c0b3db0 0x7f945c09da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9463e8d640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f945c1459f0 con 0x7f945c06e410 2024-04-27T07:05:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9463e8d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f945c144a80 con 0x7f945c0b3db0 2024-04-27T07:05:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9463e8d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f945c0b4ec0 con 0x7f945c0b4ae0 2024-04-27T07:05:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9461428640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f945c0b3db0 0x7f945c09da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9461428640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f945c0b3db0 0x7f945c09da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35080/0 (socket says 172.21.15.12:35080) 2024-04-27T07:05:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9461428640 1 -- 172.21.15.12:0/633960772 learned_addr learned my addr 172.21.15.12:0/633960772 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9461c29640 1 --2- 172.21.15.12:0/633960772 >> v2:172.21.15.53:3300/0 conn(0x7f945c0b4ae0 0x7f945c1454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9461428640 1 -- 172.21.15.12:0/633960772 >> v2:172.21.15.53:3300/0 conn(0x7f945c0b4ae0 msgr2=0x7f945c1454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9461428640 1 --2- 172.21.15.12:0/633960772 >> v2:172.21.15.53:3300/0 conn(0x7f945c0b4ae0 0x7f945c1454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9461428640 1 -- 172.21.15.12:0/633960772 >> v1:172.21.15.53:6789/0 conn(0x7f945c06e410 legacy=0x7f945c147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.068+0000 7f9461428640 1 -- 172.21.15.12:0/633960772 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c144cb0 con 0x7f945c0b3db0 2024-04-27T07:05:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461428640 1 --2- 172.21.15.12:0/633960772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c0b3db0 0x7f945c09da80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f945c09c170 tx=0x7f944c01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139be1d06f9d4d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9460c27640 1 -- 172.21.15.12:0/633960772 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c0b4070 con 0x7f945c0b3db0 2024-04-27T07:05:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9460c27640 1 -- 172.21.15.12:0/633960772 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944c0a7b90 con 0x7f945c0b3db0 2024-04-27T07:05:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9460c27640 1 -- 172.21.15.12:0/633960772 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c0af440 con 0x7f945c0b3db0 2024-04-27T07:05:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- 172.21.15.12:0/633960772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c0b3db0 msgr2=0x7f945c09da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 --2- 172.21.15.12:0/633960772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c0b3db0 0x7f945c09da80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f945c09c170 tx=0x7f944c01ebb0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- 172.21.15.12:0/633960772 shutdown_connections 2024-04-27T07:05:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 --2- 172.21.15.12:0/633960772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c0b3db0 0x7f945c09da80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 --2- 172.21.15.12:0/633960772 >> v2:172.21.15.53:3300/0 conn(0x7f945c0b4ae0 0x7f945c1454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- 172.21.15.12:0/633960772 >> 172.21.15.12:0/633960772 conn(0x7f945c0b93b0 msgr2=0x7f945c0b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- 172.21.15.12:0/633960772 shutdown_connections 2024-04-27T07:05:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- 172.21.15.12:0/633960772 wait complete. 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 Processor -- start 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- start start 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c06e410 0x7f945c13e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f945c0b3db0 0x7f945c1dc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f945c0b4ae0 0x7f945c1dcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f945c06d480 con 0x7f945c06e410 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f945c06d300 con 0x7f945c0b3db0 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f945c06d600 con 0x7f945c0b4ae0 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461c29640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c06e410 0x7f945c13e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461c29640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c06e410 0x7f945c13e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35092/0 (socket says 172.21.15.12:35092) 2024-04-27T07:05:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461c29640 1 -- 172.21.15.12:0/1274228428 learned_addr learned my addr 172.21.15.12:0/1274228428 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461428640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f945c0b3db0 0x7f945c1dc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461c29640 1 -- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f945c0b4ae0 msgr2=0x7f945c1dcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f946242a640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f945c0b4ae0 0x7f945c1dcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461c29640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f945c0b4ae0 0x7f945c1dcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461c29640 1 -- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f945c0b3db0 msgr2=0x7f945c1dc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461c29640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f945c0b3db0 0x7f945c1dc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461c29640 1 -- 172.21.15.12:0/1274228428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c070f80 con 0x7f945c06e410 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9461c29640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c06e410 0x7f945c13e000 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f945800d620 tx=0x7f945800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f94527fc640 1 -- 172.21.15.12:0/1274228428 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945800b620 con 0x7f945c06e410 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f94527fc640 1 -- 172.21.15.12:0/1274228428 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9458019070 con 0x7f945c06e410 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- 172.21.15.12:0/1274228428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945c13e6a0 con 0x7f945c06e410 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f94527fc640 1 -- 172.21.15.12:0/1274228428 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945800b9e0 con 0x7f945c06e410 2024-04-27T07:05:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.072+0000 7f9463e8d640 1 -- 172.21.15.12:0/1274228428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945c13d250 con 0x7f945c06e410 2024-04-27T07:05:11.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.076+0000 7f9463e8d640 1 -- 172.21.15.12:0/1274228428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f945c068c20 con 0x7f945c06e410 2024-04-27T07:05:11.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.076+0000 7f94527fc640 1 -- 172.21.15.12:0/1274228428 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f945800bb80 con 0x7f945c06e410 2024-04-27T07:05:11.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.076+0000 7f94527fc640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9440068380 0x7f944006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.076+0000 7f94527fc640 1 -- 172.21.15.12:0/1274228428 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f9458086dc0 con 0x7f945c06e410 2024-04-27T07:05:11.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.076+0000 7f9461428640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9440068380 0x7f944006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.076+0000 7f9461428640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9440068380 0x7f944006a840 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f944c0b1000 tx=0x7f944c0a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.080+0000 7f94527fc640 1 -- 172.21.15.12:0/1274228428 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9458057360 con 0x7f945c06e410 2024-04-27T07:05:11.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.244+0000 7f9463e8d640 1 -- 172.21.15.12:0/1274228428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f945c09dc90 con 0x7f945c06e410 2024-04-27T07:05:11.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.244+0000 7f94527fc640 1 -- 172.21.15.12:0/1274228428 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f9458056c60 con 0x7f945c06e410 2024-04-27T07:05:11.251 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:05:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 -- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9440068380 msgr2=0x7f944006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9440068380 0x7f944006a840 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f944c0b1000 tx=0x7f944c0a9070 comp rx=0 tx=0).stop 2024-04-27T07:05:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 -- 172.21.15.12:0/1274228428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c06e410 msgr2=0x7f945c13e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c06e410 0x7f945c13e000 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f945800d620 tx=0x7f945800dae0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 -- 172.21.15.12:0/1274228428 shutdown_connections 2024-04-27T07:05:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9440068380 0x7f944006a840 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f945c0b4ae0 0x7f945c1dcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f945c0b3db0 0x7f945c1dc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 --2- 172.21.15.12:0/1274228428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f945c06e410 0x7f945c13e000 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 -- 172.21.15.12:0/1274228428 >> 172.21.15.12:0/1274228428 conn(0x7f945c0b93b0 msgr2=0x7f945c1429f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 -- 172.21.15.12:0/1274228428 shutdown_connections 2024-04-27T07:05:11.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.248+0000 7f9463e8d640 1 -- 172.21.15.12:0/1274228428 wait complete. 2024-04-27T07:05:11.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:05:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.396+0000 7f03c15c5640 1 Processor -- start 2024-04-27T07:05:11.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.396+0000 7f03c15c5640 1 -- start start 2024-04-27T07:05:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03c15c5640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f03bc0b9c20 0x7f03bc0ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03c15c5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03bc06ea10 con 0x7f03bc1495b0 2024-04-27T07:05:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03c15c5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03bc145b80 con 0x7f03bc13fb40 2024-04-27T07:05:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03c15c5640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f03bc149e60 con 0x7f03bc0b9c20 2024-04-27T07:05:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03ba7fc640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f03bc1495b0 0x7f03bc146360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56254/0 (socket says 172.21.15.12:56254) 2024-04-27T07:05:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03ba7fc640 1 -- 172.21.15.12:0/151825270 learned_addr learned my addr 172.21.15.12:0/151825270 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03baffd640 1 --2- 172.21.15.12:0/151825270 >> v2:172.21.15.53:3300/0 conn(0x7f03bc0b9c20 0x7f03bc0ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 658879696 0 0) 0x7f03bc06ea10 con 0x7f03bc1495b0 2024-04-27T07:05:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03a4003530 con 0x7f03bc1495b0 2024-04-27T07:05:11.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2652438313 0 0) 0x7f03a4003530 con 0x7f03bc1495b0 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 >> v2:172.21.15.53:3300/0 conn(0x7f03bc0b9c20 msgr2=0x7f03bc0ba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 --2- 172.21.15.12:0/151825270 >> v2:172.21.15.53:3300/0 conn(0x7f03bc0b9c20 0x7f03bc0ba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 >> v1:172.21.15.53:6789/0 conn(0x7f03bc13fb40 legacy=0x7f03bc150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc145e40 con 0x7f03bc1495b0 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f03b00016a0 con 0x7f03bc1495b0 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 >> v1:172.21.15.12:6789/0 conn(0x7f03bc1495b0 legacy=0x7f03bc146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a40039c0 0x7f03a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a4004810 0x7f03a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03a40052f0 0x7f03a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc149e60 con 0x7f03a40039c0 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f03a400c1c0 con 0x7f03a4004810 2024-04-27T07:05:11.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f03a400c3c0 con 0x7f03a40052f0 2024-04-27T07:05:11.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03baffd640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a40039c0 0x7f03a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03ba7fc640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a4004810 0x7f03a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03baffd640 1 -- 172.21.15.12:0/151825270 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03a40052f0 msgr2=0x7f03a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:11.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03baffd640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03a40052f0 0x7f03a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03baffd640 1 -- 172.21.15.12:0/151825270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a4004810 msgr2=0x7f03a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:11.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03baffd640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a4004810 0x7f03a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03baffd640 1 -- 172.21.15.12:0/151825270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a4003530 con 0x7f03a40039c0 2024-04-27T07:05:11.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03bb7fe640 1 -- 172.21.15.12:0/151825270 reap_dead start 2024-04-27T07:05:11.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03ba7fc640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03a4004810 0x7f03a4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03baffd640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a40039c0 0x7f03a4003de0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009ab0 tx=0x7f03ac01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cc3038f5b6e7b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0096b0 con 0x7f03a40039c0 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac0a6410 con 0x7f03a40039c0 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.400+0000 7f03b9ffb640 1 -- 172.21.15.12:0/151825270 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0a6800 con 0x7f03a40039c0 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- 172.21.15.12:0/151825270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a40039c0 msgr2=0x7f03a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a40039c0 0x7f03a4003de0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009ab0 tx=0x7f03ac01ca30 comp rx=0 tx=0).stop 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- 172.21.15.12:0/151825270 shutdown_connections 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 --2- 172.21.15.12:0/151825270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a40039c0 0x7f03a4003de0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- 172.21.15.12:0/151825270 >> 172.21.15.12:0/151825270 conn(0x7f03bc0ba740 msgr2=0x7f03bc0bab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- 172.21.15.12:0/151825270 shutdown_connections 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- 172.21.15.12:0/151825270 wait complete. 2024-04-27T07:05:11.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 Processor -- start 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- start start 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03a40039c0 0x7f03bc1e2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc099c00 0x7f03bc0990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03bc099610 0x7f03bc0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc06bf50 con 0x7f03bc099610 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc06bdd0 con 0x7f03bc099c00 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f03bc06c0d0 con 0x7f03a40039c0 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03bb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03bc099610 0x7f03bc0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03bb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03bc099610 0x7f03bc0b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35112/0 (socket says 172.21.15.12:35112) 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03bb7fe640 1 -- 172.21.15.12:0/499995887 learned_addr learned my addr 172.21.15.12:0/499995887 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03ba7fc640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc099c00 0x7f03bc0990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03baffd640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03a40039c0 0x7f03bc1e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03bb7fe640 1 -- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03a40039c0 msgr2=0x7f03bc1e2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03bb7fe640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03a40039c0 0x7f03bc1e2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03bb7fe640 1 -- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc099c00 msgr2=0x7f03bc0990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03bb7fe640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc099c00 0x7f03bc0990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03bb7fe640 1 -- 172.21.15.12:0/499995887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc0b26a0 con 0x7f03bc099610 2024-04-27T07:05:11.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03bb7fe640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03bc099610 0x7f03bc0b3a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f03bc068290 tx=0x7f03b400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:11.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03a37fe640 1 -- 172.21.15.12:0/499995887 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4017070 con 0x7f03bc099610 2024-04-27T07:05:11.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03a37fe640 1 -- 172.21.15.12:0/499995887 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b4005d70 con 0x7f03bc099610 2024-04-27T07:05:11.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- 172.21.15.12:0/499995887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03bc0b2f40 con 0x7f03bc099610 2024-04-27T07:05:11.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03a37fe640 1 -- 172.21.15.12:0/499995887 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b400a7f0 con 0x7f03bc099610 2024-04-27T07:05:11.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.404+0000 7f03c15c5640 1 -- 172.21.15.12:0/499995887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03bc0b2910 con 0x7f03bc099610 2024-04-27T07:05:11.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.408+0000 7f03c15c5640 1 -- 172.21.15.12:0/499995887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03bc069330 con 0x7f03bc099610 2024-04-27T07:05:11.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.408+0000 7f03a37fe640 1 -- 172.21.15.12:0/499995887 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f03b401a070 con 0x7f03bc099610 2024-04-27T07:05:11.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.408+0000 7f03a37fe640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f03a8068380 0x7f03a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.408+0000 7f03a37fe640 1 -- 172.21.15.12:0/499995887 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f03b4086140 con 0x7f03bc099610 2024-04-27T07:05:11.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.408+0000 7f03baffd640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f03a8068380 0x7f03a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.408+0000 7f03baffd640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f03a8068380 0x7f03a806a840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0ad000 tx=0x7f03ac0a9040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:11.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.412+0000 7f03a37fe640 1 -- 172.21.15.12:0/499995887 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f03b4021020 con 0x7f03bc099610 2024-04-27T07:05:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.568+0000 7f03c15c5640 1 -- 172.21.15.12:0/499995887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f03bc06f810 con 0x7f03bc099610 2024-04-27T07:05:11.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03a37fe640 1 -- 172.21.15.12:0/499995887 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f03b40566e0 con 0x7f03bc099610 2024-04-27T07:05:11.576 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:05:11.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 -- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f03a8068380 msgr2=0x7f03a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f03a8068380 0x7f03a806a840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0ad000 tx=0x7f03ac0a9040 comp rx=0 tx=0).stop 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 -- 172.21.15.12:0/499995887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03bc099610 msgr2=0x7f03bc0b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03bc099610 0x7f03bc0b3a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f03bc068290 tx=0x7f03b400cb90 comp rx=0 tx=0).stop 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 -- 172.21.15.12:0/499995887 shutdown_connections 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f03a8068380 0x7f03a806a840 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03bc099610 0x7f03bc0b3a20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc099c00 0x7f03bc0990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 --2- 172.21.15.12:0/499995887 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f03a40039c0 0x7f03bc1e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 -- 172.21.15.12:0/499995887 >> 172.21.15.12:0/499995887 conn(0x7f03bc0ba740 msgr2=0x7f03bc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 -- 172.21.15.12:0/499995887 shutdown_connections 2024-04-27T07:05:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.572+0000 7f03c15c5640 1 -- 172.21.15.12:0/499995887 wait complete. 2024-04-27T07:05:11.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:05:11.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.712+0000 7facc566d640 1 Processor -- start 2024-04-27T07:05:11.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.712+0000 7facc566d640 1 -- start start 2024-04-27T07:05:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7facc00b69a0 0x7facc0144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7facc0141670 0x7facc0141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facc0144870 con 0x7facc009d440 2024-04-27T07:05:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7facc0142010 con 0x7facc0141670 2024-04-27T07:05:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7facc00b58d0 con 0x7facc00b69a0 2024-04-27T07:05:11.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbe7fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7facc0141670 0x7facc0141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbf7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7facc009d440 0x7facc009d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56256/0 (socket says 172.21.15.12:56256) 2024-04-27T07:05:11.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbf7fe640 1 -- 172.21.15.12:0/1462874348 learned_addr learned my addr 172.21.15.12:0/1462874348 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:11.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbeffd640 1 --2- 172.21.15.12:0/1462874348 >> v2:172.21.15.53:3300/0 conn(0x7facc00b69a0 0x7facc0144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbe7fc640 1 -- 172.21.15.12:0/1462874348 >> v2:172.21.15.53:3300/0 conn(0x7facc00b69a0 msgr2=0x7facc0144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:11.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbe7fc640 1 --2- 172.21.15.12:0/1462874348 >> v2:172.21.15.53:3300/0 conn(0x7facc00b69a0 0x7facc0144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbe7fc640 1 -- 172.21.15.12:0/1462874348 >> v1:172.21.15.12:6789/0 conn(0x7facc009d440 legacy=0x7facc009d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbe7fc640 1 -- 172.21.15.12:0/1462874348 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc00b6d80 con 0x7facc0141670 2024-04-27T07:05:11.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbeffd640 1 --2- 172.21.15.12:0/1462874348 >> v2:172.21.15.53:3300/0 conn(0x7facc00b69a0 0x7facc0144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:11.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbe7fc640 1 --2- 172.21.15.12:0/1462874348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc0141670 0x7facc0141ad0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facc0069d80 tx=0x7faca801eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb48ac7c53481d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:11.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbdffb640 1 -- 172.21.15.12:0/1462874348 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca80b4070 con 0x7facc0141670 2024-04-27T07:05:11.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbdffb640 1 -- 172.21.15.12:0/1462874348 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca80a7af0 con 0x7facc0141670 2024-04-27T07:05:11.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facbdffb640 1 -- 172.21.15.12:0/1462874348 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca80af440 con 0x7facc0141670 2024-04-27T07:05:11.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 -- 172.21.15.12:0/1462874348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc0141670 msgr2=0x7facc0141ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 --2- 172.21.15.12:0/1462874348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc0141670 0x7facc0141ad0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7facc0069d80 tx=0x7faca801eaa0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 -- 172.21.15.12:0/1462874348 shutdown_connections 2024-04-27T07:05:11.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 --2- 172.21.15.12:0/1462874348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc0141670 0x7facc0141ad0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 --2- 172.21.15.12:0/1462874348 >> v2:172.21.15.53:3300/0 conn(0x7facc00b69a0 0x7facc0144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 -- 172.21.15.12:0/1462874348 >> 172.21.15.12:0/1462874348 conn(0x7facc00b93b0 msgr2=0x7facc00b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:11.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 -- 172.21.15.12:0/1462874348 shutdown_connections 2024-04-27T07:05:11.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.716+0000 7facc566d640 1 -- 172.21.15.12:0/1462874348 wait complete. 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 Processor -- start 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 -- start start 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7facc009d440 0x7facc0136590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc00b69a0 0x7facc01427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc0141670 0x7facc0142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7facc00b9ea0 con 0x7facc00b69a0 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7facc0144ea0 con 0x7facc0141670 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7facc00ba020 con 0x7facc009d440 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facbe7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc00b69a0 0x7facc01427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facbe7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc00b69a0 0x7facc01427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35132/0 (socket says 172.21.15.12:35132) 2024-04-27T07:05:11.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facbe7fc640 1 -- 172.21.15.12:0/3868112959 learned_addr learned my addr 172.21.15.12:0/3868112959 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facbe7fc640 1 -- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7facc009d440 msgr2=0x7facc0136590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facbe7fc640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7facc009d440 0x7facc0136590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facbe7fc640 1 -- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc0141670 msgr2=0x7facc0142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facbe7fc640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc0141670 0x7facc0142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facbe7fc640 1 -- 172.21.15.12:0/3868112959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc00b0b30 con 0x7facc00b69a0 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facbe7fc640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc00b69a0 0x7facc01427f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faca801c6d0 tx=0x7faca801b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facb77fe640 1 -- 172.21.15.12:0/3868112959 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca80b4040 con 0x7facc00b69a0 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facb77fe640 1 -- 172.21.15.12:0/3868112959 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca80bd070 con 0x7facc00b69a0 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 -- 172.21.15.12:0/3868112959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc00b72b0 con 0x7facc00b69a0 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facb77fe640 1 -- 172.21.15.12:0/3868112959 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca80afe20 con 0x7facc00b69a0 2024-04-27T07:05:11.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 -- 172.21.15.12:0/3868112959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc00b07b0 con 0x7facc00b69a0 2024-04-27T07:05:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facc566d640 1 -- 172.21.15.12:0/3868112959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc0068c20 con 0x7facc00b69a0 2024-04-27T07:05:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.720+0000 7facb77fe640 1 -- 172.21.15.12:0/3868112959 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faca80c2460 con 0x7facc00b69a0 2024-04-27T07:05:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.724+0000 7facb77fe640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faca0068380 0x7faca006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.724+0000 7facb77fe640 1 -- 172.21.15.12:0/3868112959 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7faca81214c0 con 0x7facc00b69a0 2024-04-27T07:05:11.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.724+0000 7facbeffd640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faca0068380 0x7faca006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:11.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.724+0000 7facbeffd640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faca0068380 0x7faca006a840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7facb000a8d0 tx=0x7facb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:11.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.728+0000 7facb77fe640 1 -- 172.21.15.12:0/3868112959 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faca8018720 con 0x7facc00b69a0 2024-04-27T07:05:11.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.888+0000 7facc566d640 1 -- 172.21.15.12:0/3868112959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7facc00b3850 con 0x7facc00b69a0 2024-04-27T07:05:11.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.888+0000 7facb77fe640 1 -- 172.21.15.12:0/3868112959 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7faca80f1b10 con 0x7facc00b69a0 2024-04-27T07:05:11.894 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:05:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 -- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faca0068380 msgr2=0x7faca006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faca0068380 0x7faca006a840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7facb000a8d0 tx=0x7facb0008040 comp rx=0 tx=0).stop 2024-04-27T07:05:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 -- 172.21.15.12:0/3868112959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc00b69a0 msgr2=0x7facc01427f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc00b69a0 0x7facc01427f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faca801c6d0 tx=0x7faca801b840 comp rx=0 tx=0).stop 2024-04-27T07:05:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 -- 172.21.15.12:0/3868112959 shutdown_connections 2024-04-27T07:05:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7faca0068380 0x7faca006a840 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7facc0141670 0x7facc0142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7facc00b69a0 0x7facc01427f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 --2- 172.21.15.12:0/3868112959 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7facc009d440 0x7facc0136590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 -- 172.21.15.12:0/3868112959 >> 172.21.15.12:0/3868112959 conn(0x7facc00b93b0 msgr2=0x7facc009b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 -- 172.21.15.12:0/3868112959 shutdown_connections 2024-04-27T07:05:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:11.892+0000 7facc566d640 1 -- 172.21.15.12:0/3868112959 wait complete. 2024-04-27T07:05:11.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T07:05:12.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d79f93640 1 Processor -- start 2024-04-27T07:05:12.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d79f93640 1 -- start start 2024-04-27T07:05:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d79f93640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d740b2ae0 0x7f8d740b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d79f93640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d74068870 con 0x7f8d7414e4d0 2024-04-27T07:05:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d79f93640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d74067f50 con 0x7f8d74067b70 2024-04-27T07:05:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d79f93640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8d740b2650 con 0x7f8d740b2ae0 2024-04-27T07:05:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d73fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8d7414e4d0 0x7f8d741508c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56270/0 (socket says 172.21.15.12:56270) 2024-04-27T07:05:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d73fff640 1 -- 172.21.15.12:0/1834420776 learned_addr learned my addr 172.21.15.12:0/1834420776 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:12.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d72ffd640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d740b2ae0 0x7f8d740b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3752465250 0 0) 0x7f8d74068870 con 0x7f8d7414e4d0 2024-04-27T07:05:12.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d60003530 con 0x7f8d7414e4d0 2024-04-27T07:05:12.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3968849313 0 0) 0x7f8d60003530 con 0x7f8d7414e4d0 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d740b2ae0 msgr2=0x7f8d740b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d740b2ae0 0x7f8d740b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 >> v1:172.21.15.53:6789/0 conn(0x7f8d74067b70 legacy=0x7f8d74143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d74072f70 con 0x7f8d7414e4d0 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f8d6c001680 con 0x7f8d7414e4d0 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 >> v1:172.21.15.12:6789/0 conn(0x7f8d7414e4d0 legacy=0x7f8d741508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d600039f0 0x7f8d60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d60004810 0x7f8d60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d740b2ae0 0x7f8d60005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d740b2650 con 0x7f8d600039f0 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6000bb00 con 0x7f8d60004810 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6000bd00 con 0x7f8d740b2ae0 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d737fe640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d600039f0 0x7f8d60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d737fe640 1 -- 172.21.15.12:0/1834420776 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d740b2ae0 msgr2=0x7f8d60005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d737fe640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d740b2ae0 0x7f8d60005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d72ffd640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d60004810 0x7f8d60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d737fe640 1 -- 172.21.15.12:0/1834420776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d60004810 msgr2=0x7f8d60004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:12.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d737fe640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d60004810 0x7f8d60004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.028+0000 7f8d737fe640 1 -- 172.21.15.12:0/1834420776 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d60003530 con 0x7f8d600039f0 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d737fe640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d600039f0 0x7f8d60003e10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8d64001780 tx=0x7f8d6401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8c92b719b8cd1b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d640013a0 con 0x7f8d600039f0 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d640052f0 con 0x7f8d600039f0 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d727fc640 1 -- 172.21.15.12:0/1834420776 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64003840 con 0x7f8d600039f0 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d79f93640 1 -- 172.21.15.12:0/1834420776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d600039f0 msgr2=0x7f8d60003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d79f93640 1 --2- 172.21.15.12:0/1834420776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d600039f0 0x7f8d60003e10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8d64001780 tx=0x7f8d6401c400 comp rx=0 tx=0).stop 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d73fff640 1 -- 172.21.15.12:0/1834420776 reap_dead start 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d79f93640 1 -- 172.21.15.12:0/1834420776 shutdown_connections 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d79f93640 1 -- 172.21.15.12:0/1834420776 >> 172.21.15.12:0/1834420776 conn(0x7f8d740b67b0 msgr2=0x7f8d740b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d79f93640 1 -- 172.21.15.12:0/1834420776 shutdown_connections 2024-04-27T07:05:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d79f93640 1 -- 172.21.15.12:0/1834420776 wait complete. 2024-04-27T07:05:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d79f93640 1 Processor -- start 2024-04-27T07:05:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.032+0000 7f8d79f93640 1 -- start start 2024-04-27T07:05:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d79f93640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d600039f0 0x7f8d7413fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d79f93640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7413a520 0x7f8d74143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d79f93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d74143ff0 0x7f8d74138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d79f93640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d740b3a90 con 0x7f8d7413a520 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d79f93640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d740b6520 con 0x7f8d74143ff0 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d79f93640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8d740b72d0 con 0x7f8d600039f0 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d72ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7413a520 0x7f8d74143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d72ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7413a520 0x7f8d74143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35154/0 (socket says 172.21.15.12:35154) 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d72ffd640 1 -- 172.21.15.12:0/192197936 learned_addr learned my addr 172.21.15.12:0/192197936 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d737fe640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d600039f0 0x7f8d7413fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d73fff640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d74143ff0 0x7f8d74138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d72ffd640 1 -- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d600039f0 msgr2=0x7f8d7413fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d72ffd640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d600039f0 0x7f8d7413fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d72ffd640 1 -- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d74143ff0 msgr2=0x7f8d74138f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d72ffd640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d74143ff0 0x7f8d74138f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d72ffd640 1 -- 172.21.15.12:0/192197936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d74145910 con 0x7f8d7413a520 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d72ffd640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7413a520 0x7f8d74143ab0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00f350 tx=0x7f8d5c00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d6bfff640 1 -- 172.21.15.12:0/192197936 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c01c070 con 0x7f8d7413a520 2024-04-27T07:05:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d6bfff640 1 -- 172.21.15.12:0/192197936 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5c017620 con 0x7f8d7413a520 2024-04-27T07:05:12.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d6bfff640 1 -- 172.21.15.12:0/192197936 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c017a10 con 0x7f8d7413a520 2024-04-27T07:05:12.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d79f93640 1 -- 172.21.15.12:0/192197936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d74145bf0 con 0x7f8d7413a520 2024-04-27T07:05:12.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d79f93640 1 -- 172.21.15.12:0/192197936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d74144cb0 con 0x7f8d7413a520 2024-04-27T07:05:12.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d79f93640 1 -- 172.21.15.12:0/192197936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d74069330 con 0x7f8d7413a520 2024-04-27T07:05:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d6bfff640 1 -- 172.21.15.12:0/192197936 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8d5c01f070 con 0x7f8d7413a520 2024-04-27T07:05:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d6bfff640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d54068380 0x7f8d5406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d6bfff640 1 -- 172.21.15.12:0/192197936 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f8d5c0871c0 con 0x7f8d7413a520 2024-04-27T07:05:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.036+0000 7f8d737fe640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d54068380 0x7f8d5406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.040+0000 7f8d737fe640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d54068380 0x7f8d5406a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d64001750 tx=0x7f8d640011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:12.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.040+0000 7f8d6bfff640 1 -- 172.21.15.12:0/192197936 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8d5c012070 con 0x7f8d7413a520 2024-04-27T07:05:12.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.212+0000 7f8d79f93640 1 -- 172.21.15.12:0/192197936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8d740b70b0 con 0x7f8d7413a520 2024-04-27T07:05:12.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d6bfff640 1 -- 172.21.15.12:0/192197936 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f8d5c057360 con 0x7f8d7413a520 2024-04-27T07:05:12.221 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 -- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d54068380 msgr2=0x7f8d5406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d54068380 0x7f8d5406a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d64001750 tx=0x7f8d640011a0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 -- 172.21.15.12:0/192197936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7413a520 msgr2=0x7f8d74143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7413a520 0x7f8d74143ab0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00f350 tx=0x7f8d5c00f810 comp rx=0 tx=0).stop 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 -- 172.21.15.12:0/192197936 shutdown_connections 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8d54068380 0x7f8d5406a840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d74143ff0 0x7f8d74138f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d7413a520 0x7f8d74143ab0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 --2- 172.21.15.12:0/192197936 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8d600039f0 0x7f8d7413fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 -- 172.21.15.12:0/192197936 >> 172.21.15.12:0/192197936 conn(0x7f8d740b67b0 msgr2=0x7f8d74141f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:12.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 -- 172.21.15.12:0/192197936 shutdown_connections 2024-04-27T07:05:12.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.216+0000 7f8d79f93640 1 -- 172.21.15.12:0/192197936 wait complete. 2024-04-27T07:05:12.240 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi012.front.sepia.ceph.com... 2024-04-27T07:05:12.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:12.240 DEBUG:teuthology.orchestra.run.smithi012:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-27T07:05:12.251 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi012.stderr:ceph-fuse[36912]: fuse finished with error 0 and tester_r 0 2024-04-27T07:05:12.253 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:05:12.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:12.253 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-27T07:05:12.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:05:12.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83da556640 1 Processor -- start 2024-04-27T07:05:12.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83da556640 1 -- start start 2024-04-27T07:05:12.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83da556640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40b9a10 0x7f83d40b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83da556640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d406dac0 con 0x7f83d414e400 2024-04-27T07:05:12.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83da556640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d406e890 con 0x7f83d40b3680 2024-04-27T07:05:12.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83da556640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f83d40b9df0 con 0x7f83d40b9a10 2024-04-27T07:05:12.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d8af3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f83d414e400 0x7f83d41507f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56286/0 (socket says 172.21.15.12:56286) 2024-04-27T07:05:12.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d8af3640 1 -- 172.21.15.12:0/3676171387 learned_addr learned my addr 172.21.15.12:0/3676171387 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d3fff640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40b9a10 0x7f83d40b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3707791076 0 0) 0x7f83cc0014a0 con 0x7f83d414e400 2024-04-27T07:05:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83bc003530 con 0x7f83d414e400 2024-04-27T07:05:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 685501081 0 0) 0x7f83bc003530 con 0x7f83d414e400 2024-04-27T07:05:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40b9a10 msgr2=0x7f83d40b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40b9a10 0x7f83d40b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 >> v1:172.21.15.53:6789/0 conn(0x7f83d40b3680 legacy=0x7f83d4145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d406e090 con 0x7f83d414e400 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f83cc001bd0 con 0x7f83d414e400 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 >> v1:172.21.15.12:6789/0 conn(0x7f83d414e400 legacy=0x7f83d41507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 0x7f83bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40b9a10 0x7f83bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc004d50 0x7f83bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40b9df0 con 0x7f83bc0039f0 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc00bb00 con 0x7f83bc004d50 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f83bc00bd00 con 0x7f83d40b9a10 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d3fff640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 0x7f83bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d37fe640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40b9a10 0x7f83bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d8af3640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc004d50 0x7f83bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d3fff640 1 -- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40b9a10 msgr2=0x7f83bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d3fff640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40b9a10 0x7f83bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d3fff640 1 -- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc004d50 msgr2=0x7f83bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d3fff640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc004d50 0x7f83bc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d3fff640 1 -- 172.21.15.12:0/3676171387 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc003530 con 0x7f83bc0039f0 2024-04-27T07:05:12.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.528+0000 7f83d37fe640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40b9a10 0x7f83bc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83d3fff640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 0x7f83bc003e10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009ae0 tx=0x7f83c401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4420d0bf7c4fa256 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40096b0 con 0x7f83bc0039f0 2024-04-27T07:05:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c40af070 con 0x7f83bc0039f0 2024-04-27T07:05:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83d2ffd640 1 -- 172.21.15.12:0/3676171387 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40aa690 con 0x7f83bc0039f0 2024-04-27T07:05:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 -- 172.21.15.12:0/3676171387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 msgr2=0x7f83bc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 --2- 172.21.15.12:0/3676171387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 0x7f83bc003e10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009ae0 tx=0x7f83c401c9b0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83d8af3640 1 -- 172.21.15.12:0/3676171387 reap_dead start 2024-04-27T07:05:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 -- 172.21.15.12:0/3676171387 shutdown_connections 2024-04-27T07:05:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 -- 172.21.15.12:0/3676171387 >> 172.21.15.12:0/3676171387 conn(0x7f83d4067d10 msgr2=0x7f83d4068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 -- 172.21.15.12:0/3676171387 shutdown_connections 2024-04-27T07:05:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 -- 172.21.15.12:0/3676171387 wait complete. 2024-04-27T07:05:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 Processor -- start 2024-04-27T07:05:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 -- start start 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 0x7f83d409cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40af7d0 0x7f83d41de4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83d40b7a00 0x7f83d4148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4074020 con 0x7f83bc0039f0 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4073ea0 con 0x7f83d40b7a00 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83da556640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f83d4151f10 con 0x7f83d40af7d0 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 0x7f83d409cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 0x7f83d409cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35160/0 (socket says 172.21.15.12:35160) 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.532+0000 7f83d3fff640 1 -- 172.21.15.12:0/3677448859 learned_addr learned my addr 172.21.15.12:0/3677448859 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d8af3640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83d40b7a00 0x7f83d4148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d37fe640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40af7d0 0x7f83d41de4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d3fff640 1 -- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40af7d0 msgr2=0x7f83d41de4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d3fff640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40af7d0 0x7f83d41de4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d3fff640 1 -- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83d40b7a00 msgr2=0x7f83d4148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d3fff640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83d40b7a00 0x7f83d4148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d3fff640 1 -- 172.21.15.12:0/3677448859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40b8920 con 0x7f83bc0039f0 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d8af3640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83d40b7a00 0x7f83d4148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d37fe640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40af7d0 0x7f83d41de4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d3fff640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 0x7f83d409cca0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f83c40ac040 tx=0x7f83bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d0ff9640 1 -- 172.21.15.12:0/3677448859 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4006200 con 0x7f83bc0039f0 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d0ff9640 1 -- 172.21.15.12:0/3677448859 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c40a6040 con 0x7f83bc0039f0 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83da556640 1 -- 172.21.15.12:0/3677448859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d409c4a0 con 0x7f83bc0039f0 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d0ff9640 1 -- 172.21.15.12:0/3677448859 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40065e0 con 0x7f83bc0039f0 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83da556640 1 -- 172.21.15.12:0/3677448859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d41dea30 con 0x7f83bc0039f0 2024-04-27T07:05:12.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83da556640 1 -- 172.21.15.12:0/3677448859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d4068c20 con 0x7f83bc0039f0 2024-04-27T07:05:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d0ff9640 1 -- 172.21.15.12:0/3677448859 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f83c4006780 con 0x7f83bc0039f0 2024-04-27T07:05:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d0ff9640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f83c00683b0 0x7f83c006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d0ff9640 1 -- 172.21.15.12:0/3677448859 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f83c4121400 con 0x7f83bc0039f0 2024-04-27T07:05:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.536+0000 7f83d37fe640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f83c00683b0 0x7f83c006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.540+0000 7f83d37fe640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f83c00683b0 0x7f83c006a870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f83c8000c10 tx=0x7f83c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:12.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.540+0000 7f83d0ff9640 1 -- 172.21.15.12:0/3677448859 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f83c40166c0 con 0x7f83bc0039f0 2024-04-27T07:05:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.740+0000 7f83da556640 1 -- 172.21.15.12:0/3677448859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83d40afbb0 con 0x7f83bc0039f0 2024-04-27T07:05:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83d0ff9640 1 -- 172.21.15.12:0/3677448859 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3531 (secure 0 0 0) 0x7f83c40f1b30 con 0x7f83bc0039f0 2024-04-27T07:05:12.749 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:12.749 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3153112684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3153112684},{"type":"v1","addr":"172.21.15.53:6835","nonce":3153112684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:05:00.918068+0000","modified":"2024-04-27T07:05:03.947324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5001":{"gid":5001,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4067812811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4067812811},{"type":"v1","addr":"172.21.15.12:6833","nonce":4067812811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5001,"qdb_cluster":[5001]},"id":2}]} 2024-04-27T07:05:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 16 2024-04-27T07:05:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 -- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f83c00683b0 msgr2=0x7f83c006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f83c00683b0 0x7f83c006a870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f83c8000c10 tx=0x7f83c8009040 comp rx=0 tx=0).stop 2024-04-27T07:05:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 -- 172.21.15.12:0/3677448859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 msgr2=0x7f83d409cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 0x7f83d409cca0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f83c40ac040 tx=0x7f83bc005840 comp rx=0 tx=0).stop 2024-04-27T07:05:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 -- 172.21.15.12:0/3677448859 shutdown_connections 2024-04-27T07:05:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f83c00683b0 0x7f83c006a870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83d40b7a00 0x7f83d4148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f83d40af7d0 0x7f83d41de4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 --2- 172.21.15.12:0/3677448859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83bc0039f0 0x7f83d409cca0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 -- 172.21.15.12:0/3677448859 >> 172.21.15.12:0/3677448859 conn(0x7f83d4067d10 msgr2=0x7f83d4143ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:12.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 -- 172.21.15.12:0/3677448859 shutdown_connections 2024-04-27T07:05:12.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.744+0000 7f83da556640 1 -- 172.21.15.12:0/3677448859 wait complete. 2024-04-27T07:05:12.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:05:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.884+0000 7f43c48f5640 1 Processor -- start 2024-04-27T07:05:12.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.884+0000 7f43c48f5640 1 -- start start 2024-04-27T07:05:12.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43c48f5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f43c0149e90 0x7f43c0073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43c48f5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c00b73e0 con 0x7f43c00b4860 2024-04-27T07:05:12.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43c48f5640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c00b62c0 con 0x7f43c00b4480 2024-04-27T07:05:12.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43c48f5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f43c00b3d00 con 0x7f43c0149e90 2024-04-27T07:05:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43bed9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f43c00b4860 0x7f43c0150580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56292/0 (socket says 172.21.15.12:56292) 2024-04-27T07:05:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43bed9d640 1 -- 172.21.15.12:0/1615577160 learned_addr learned my addr 172.21.15.12:0/1615577160 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43bdd9b640 1 --2- 172.21.15.12:0/1615577160 >> v2:172.21.15.12:3300/0 conn(0x7f43c0149e90 0x7f43c0073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43bd59a640 1 -- 172.21.15.12:0/1615577160 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1817791108 0 0) 0x7f43c00b73e0 con 0x7f43c00b4860 2024-04-27T07:05:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43bd59a640 1 -- 172.21.15.12:0/1615577160 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43ac003530 con 0x7f43c00b4860 2024-04-27T07:05:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43bdd9b640 1 -- 172.21.15.12:0/1615577160 >> v1:172.21.15.53:6789/0 conn(0x7f43c00b4480 legacy=0x7f43c0149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43bdd9b640 1 -- 172.21.15.12:0/1615577160 >> v1:172.21.15.12:6789/0 conn(0x7f43c00b4860 legacy=0x7f43c0150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.888+0000 7f43bdd9b640 1 -- 172.21.15.12:0/1615577160 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c00b2b20 con 0x7f43c0149e90 2024-04-27T07:05:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43bdd9b640 1 --2- 172.21.15.12:0/1615577160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 0x7f43c0073bc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f43a80097c0 tx=0x7f43a801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ddbcbda1b0ef09 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43bd59a640 1 -- 172.21.15.12:0/1615577160 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80b0070 con 0x7f43c0149e90 2024-04-27T07:05:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43bd59a640 1 -- 172.21.15.12:0/1615577160 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a80ab600 con 0x7f43c0149e90 2024-04-27T07:05:12.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43bd59a640 1 -- 172.21.15.12:0/1615577160 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80ab9d0 con 0x7f43c0149e90 2024-04-27T07:05:12.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43c48f5640 1 -- 172.21.15.12:0/1615577160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 msgr2=0x7f43c0073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43c48f5640 1 --2- 172.21.15.12:0/1615577160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 0x7f43c0073bc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f43a80097c0 tx=0x7f43a801c380 comp rx=0 tx=0).stop 2024-04-27T07:05:12.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43c48f5640 1 -- 172.21.15.12:0/1615577160 shutdown_connections 2024-04-27T07:05:12.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43c48f5640 1 --2- 172.21.15.12:0/1615577160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 0x7f43c0073bc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43c48f5640 1 -- 172.21.15.12:0/1615577160 >> 172.21.15.12:0/1615577160 conn(0x7f43c00bb4f0 msgr2=0x7f43c00bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43c48f5640 1 -- 172.21.15.12:0/1615577160 shutdown_connections 2024-04-27T07:05:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43c48f5640 1 -- 172.21.15.12:0/1615577160 wait complete. 2024-04-27T07:05:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43c48f5640 1 Processor -- start 2024-04-27T07:05:12.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.892+0000 7f43c48f5640 1 -- start start 2024-04-27T07:05:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43c48f5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00b4480 0x7f43c00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43c48f5640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f43c00b4860 0x7f43c00ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43c48f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 0x7f43c00abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43c48f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0154850 con 0x7f43c0149e90 2024-04-27T07:05:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43c48f5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01546d0 con 0x7f43c00b4480 2024-04-27T07:05:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43c48f5640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f43c01549d0 con 0x7f43c00b4860 2024-04-27T07:05:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43bed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 0x7f43c00abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43bed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 0x7f43c00abbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35190/0 (socket says 172.21.15.12:35190) 2024-04-27T07:05:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43bed9d640 1 -- 172.21.15.12:0/3677112314 learned_addr learned my addr 172.21.15.12:0/3677112314 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43be59c640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00b4480 0x7f43c00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43bdd9b640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f43c00b4860 0x7f43c00ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43bed9d640 1 -- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f43c00b4860 msgr2=0x7f43c00ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43bed9d640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f43c00b4860 0x7f43c00ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43bed9d640 1 -- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00b4480 msgr2=0x7f43c00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43bed9d640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00b4480 0x7f43c00adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43bed9d640 1 -- 172.21.15.12:0/3677112314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c00ae060 con 0x7f43c0149e90 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43bed9d640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 0x7f43c00abbf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c450 tx=0x7f43b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43b6ffd640 1 -- 172.21.15.12:0/3677112314 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8017070 con 0x7f43c0149e90 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43b6ffd640 1 -- 172.21.15.12:0/3677112314 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b8005cf0 con 0x7f43c0149e90 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43b6ffd640 1 -- 172.21.15.12:0/3677112314 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80072b0 con 0x7f43c0149e90 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43c48f5640 1 -- 172.21.15.12:0/3677112314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c00aae80 con 0x7f43c0149e90 2024-04-27T07:05:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43c48f5640 1 -- 172.21.15.12:0/3677112314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c01df460 con 0x7f43c0149e90 2024-04-27T07:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43c48f5640 1 -- 172.21.15.12:0/3677112314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c0068c20 con 0x7f43c0149e90 2024-04-27T07:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43b6ffd640 1 -- 172.21.15.12:0/3677112314 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f43b801a070 con 0x7f43c0149e90 2024-04-27T07:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43b6ffd640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f43a0068380 0x7f43a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43b6ffd640 1 -- 172.21.15.12:0/3677112314 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f43b8085c30 con 0x7f43c0149e90 2024-04-27T07:05:12.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.896+0000 7f43be59c640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f43a0068380 0x7f43a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:12.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.900+0000 7f43be59c640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f43a0068380 0x7f43a006a840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f43b0001d50 tx=0x7f43b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:12.900+0000 7f43b6ffd640 1 -- 172.21.15.12:0/3677112314 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f43b80561e0 con 0x7f43c0149e90 2024-04-27T07:05:13.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.056+0000 7f43c48f5640 1 -- 172.21.15.12:0/3677112314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43c00bae80 con 0x7f43c0149e90 2024-04-27T07:05:13.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.060+0000 7f43b6ffd640 1 -- 172.21.15.12:0/3677112314 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16609 (secure 0 0 0) 0x7f43b8055ae0 con 0x7f43c0149e90 2024-04-27T07:05:13.067 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:13.067 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:05:02.565909+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:59.279361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:59.893681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:05:13.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 -- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f43a0068380 msgr2=0x7f43a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f43a0068380 0x7f43a006a840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f43b0001d50 tx=0x7f43b0009040 comp rx=0 tx=0).stop 2024-04-27T07:05:13.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 -- 172.21.15.12:0/3677112314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 msgr2=0x7f43c00abbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 0x7f43c00abbf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c450 tx=0x7f43b800c910 comp rx=0 tx=0).stop 2024-04-27T07:05:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 -- 172.21.15.12:0/3677112314 shutdown_connections 2024-04-27T07:05:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f43a0068380 0x7f43a006a840 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c0149e90 0x7f43c00abbf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f43c00b4860 0x7f43c00ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 --2- 172.21.15.12:0/3677112314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43c00b4480 0x7f43c00adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 -- 172.21.15.12:0/3677112314 >> 172.21.15.12:0/3677112314 conn(0x7f43c00bb4f0 msgr2=0x7f43c00ba0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 -- 172.21.15.12:0/3677112314 shutdown_connections 2024-04-27T07:05:13.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.064+0000 7f43c48f5640 1 -- 172.21.15.12:0/3677112314 wait complete. 2024-04-27T07:05:13.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-27T07:04:08.650986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-27T07:04:59.279361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-27T07:04:59.893681+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-27T07:05:13.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T07:05:13.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d4165640 1 Processor -- start 2024-04-27T07:05:13.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d4165640 1 -- start start 2024-04-27T07:05:13.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d4165640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f55cc0b0470 0x7f55cc09cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d4165640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55cc0b0990 con 0x7f55cc0af770 2024-04-27T07:05:13.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d4165640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55cc0b0c70 con 0x7f55cc0b6a50 2024-04-27T07:05:13.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d4165640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f55cc145610 con 0x7f55cc0b0470 2024-04-27T07:05:13.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d2702640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f55cc0b0470 0x7f55cc09cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d1f01640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f55cc0af770 0x7f55cc144f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56302/0 (socket says 172.21.15.12:56302) 2024-04-27T07:05:13.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d1f01640 1 -- 172.21.15.12:0/3301734250 learned_addr learned my addr 172.21.15.12:0/3301734250 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:13.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d2702640 1 -- 172.21.15.12:0/3301734250 >> v1:172.21.15.53:6789/0 conn(0x7f55cc0b6a50 legacy=0x7f55cc09c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:13.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d0eff640 1 -- 172.21.15.12:0/3301734250 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3697033794 0 0) 0x7f55cc0b0990 con 0x7f55cc0af770 2024-04-27T07:05:13.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d2702640 1 -- 172.21.15.12:0/3301734250 >> v1:172.21.15.12:6789/0 conn(0x7f55cc0af770 legacy=0x7f55cc144f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.204+0000 7f55d2702640 1 -- 172.21.15.12:0/3301734250 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc145790 con 0x7f55cc0b0470 2024-04-27T07:05:13.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d2702640 1 --2- 172.21.15.12:0/3301734250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0b0470 0x7f55cc09cda0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009d90 tx=0x7f55c401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bf5de9ea61964e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:13.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d0eff640 1 -- 172.21.15.12:0/3301734250 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40b2070 con 0x7f55cc0b0470 2024-04-27T07:05:13.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d0eff640 1 -- 172.21.15.12:0/3301734250 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c40a5830 con 0x7f55cc0b0470 2024-04-27T07:05:13.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d0eff640 1 -- 172.21.15.12:0/3301734250 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40a5c00 con 0x7f55cc0b0470 2024-04-27T07:05:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- 172.21.15.12:0/3301734250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0b0470 msgr2=0x7f55cc09cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 --2- 172.21.15.12:0/3301734250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0b0470 0x7f55cc09cda0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009d90 tx=0x7f55c401c970 comp rx=0 tx=0).stop 2024-04-27T07:05:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- 172.21.15.12:0/3301734250 shutdown_connections 2024-04-27T07:05:13.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 --2- 172.21.15.12:0/3301734250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0b0470 0x7f55cc09cda0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- 172.21.15.12:0/3301734250 >> 172.21.15.12:0/3301734250 conn(0x7f55cc0b6ff0 msgr2=0x7f55cc0b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:13.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- 172.21.15.12:0/3301734250 shutdown_connections 2024-04-27T07:05:13.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- 172.21.15.12:0/3301734250 wait complete. 2024-04-27T07:05:13.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 Processor -- start 2024-04-27T07:05:13.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- start start 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0af770 0x7f55cc1372c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0b0470 0x7f55cc0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55cc0b6a50 0x7f55cc0b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc06bf10 con 0x7f55cc0af770 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc06bd90 con 0x7f55cc0b0470 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f55cc06c090 con 0x7f55cc0b6a50 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1f01640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0af770 0x7f55cc1372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1f01640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0af770 0x7f55cc1372c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35214/0 (socket says 172.21.15.12:35214) 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1f01640 1 -- 172.21.15.12:0/943919045 learned_addr learned my addr 172.21.15.12:0/943919045 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d2702640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55cc0b6a50 0x7f55cc0b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1700640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0b0470 0x7f55cc0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1f01640 1 -- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55cc0b6a50 msgr2=0x7f55cc0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1f01640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55cc0b6a50 0x7f55cc0b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1f01640 1 -- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0b0470 msgr2=0x7f55cc0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1f01640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0b0470 0x7f55cc0b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1f01640 1 -- 172.21.15.12:0/943919045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc139c10 con 0x7f55cc0af770 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d2702640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55cc0b6a50 0x7f55cc0b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1700640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0b0470 0x7f55cc0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d1f01640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0af770 0x7f55cc1372c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f55c800c4d0 tx=0x7f55c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55c27fc640 1 -- 172.21.15.12:0/943919045 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8018070 con 0x7f55cc0af770 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55c27fc640 1 -- 172.21.15.12:0/943919045 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c80074e0 con 0x7f55cc0af770 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- 172.21.15.12:0/943919045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc0b2b20 con 0x7f55cc0af770 2024-04-27T07:05:13.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55c27fc640 1 -- 172.21.15.12:0/943919045 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8004a30 con 0x7f55cc0af770 2024-04-27T07:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.208+0000 7f55d4165640 1 -- 172.21.15.12:0/943919045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc0b4910 con 0x7f55cc0af770 2024-04-27T07:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.212+0000 7f55d4165640 1 -- 172.21.15.12:0/943919045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55cc069330 con 0x7f55cc0af770 2024-04-27T07:05:13.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.212+0000 7f55c27fc640 1 -- 172.21.15.12:0/943919045 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f55c801b070 con 0x7f55cc0af770 2024-04-27T07:05:13.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.212+0000 7f55c27fc640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f55b00683b0 0x7f55b006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.212+0000 7f55c27fc640 1 -- 172.21.15.12:0/943919045 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f55c8085fd0 con 0x7f55cc0af770 2024-04-27T07:05:13.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.212+0000 7f55d1700640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f55b00683b0 0x7f55b006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.212+0000 7f55d1700640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f55b00683b0 0x7f55b006a870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f55cc136ff0 tx=0x7f55bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:13.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.216+0000 7f55c27fc640 1 -- 172.21.15.12:0/943919045 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55c8056600 con 0x7f55cc0af770 2024-04-27T07:05:13.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 -- 172.21.15.12:0/943919045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f55cc09aba0 con 0x7f55cc0af770 2024-04-27T07:05:13.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55c27fc640 1 -- 172.21.15.12:0/943919045 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f55c8055f00 con 0x7f55cc0af770 2024-04-27T07:05:13.385 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 -- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f55b00683b0 msgr2=0x7f55b006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f55b00683b0 0x7f55b006a870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f55cc136ff0 tx=0x7f55bc009040 comp rx=0 tx=0).stop 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 -- 172.21.15.12:0/943919045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0af770 msgr2=0x7f55cc1372c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0af770 0x7f55cc1372c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f55c800c4d0 tx=0x7f55c800c990 comp rx=0 tx=0).stop 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 -- 172.21.15.12:0/943919045 shutdown_connections 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f55b00683b0 0x7f55b006a870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f55cc0b6a50 0x7f55cc0b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55cc0b0470 0x7f55cc0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 --2- 172.21.15.12:0/943919045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55cc0af770 0x7f55cc1372c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 -- 172.21.15.12:0/943919045 >> 172.21.15.12:0/943919045 conn(0x7f55cc0b6ff0 msgr2=0x7f55cc143fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 -- 172.21.15.12:0/943919045 shutdown_connections 2024-04-27T07:05:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.380+0000 7f55d4165640 1 -- 172.21.15.12:0/943919045 wait complete. 2024-04-27T07:05:13.405 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T07:05:13.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f76e6640 1 Processor -- start 2024-04-27T07:05:13.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f76e6640 1 -- start start 2024-04-27T07:05:13.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f76e6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f00af770 0x7f66f014b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f76e6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f66f006c140 0x7f66f014b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f76e6640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f66f00b6af0 0x7f66f014bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f76e6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f66f00b13e0 con 0x7f66f006c140 2024-04-27T07:05:13.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f76e6640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f66f00afb50 con 0x7f66f00b6af0 2024-04-27T07:05:13.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f76e6640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f66f00b1680 con 0x7f66f00af770 2024-04-27T07:05:13.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f4c81640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f66f006c140 0x7f66f014b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f4c81640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f66f006c140 0x7f66f014b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35224/0 (socket says 172.21.15.12:35224) 2024-04-27T07:05:13.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f4c81640 1 -- 172.21.15.12:0/1805304208 learned_addr learned my addr 172.21.15.12:0/1805304208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:13.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f5482640 1 --2- 172.21.15.12:0/1805304208 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f00af770 0x7f66f014b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f5c83640 1 --2- 172.21.15.12:0/1805304208 >> v2:172.21.15.53:3300/0 conn(0x7f66f00b6af0 0x7f66f014bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f4c81640 1 -- 172.21.15.12:0/1805304208 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f00af770 msgr2=0x7f66f014b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:13.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f4c81640 1 --2- 172.21.15.12:0/1805304208 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f00af770 0x7f66f014b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f4c81640 1 -- 172.21.15.12:0/1805304208 >> v2:172.21.15.53:3300/0 conn(0x7f66f00b6af0 msgr2=0x7f66f014bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:13.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f4c81640 1 --2- 172.21.15.12:0/1805304208 >> v2:172.21.15.53:3300/0 conn(0x7f66f00b6af0 0x7f66f014bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.528+0000 7f66f4c81640 1 -- 172.21.15.12:0/1805304208 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f006bc20 con 0x7f66f006c140 2024-04-27T07:05:13.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f4c81640 1 --2- 172.21.15.12:0/1805304208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f006c140 0x7f66f014b880 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66f00715a0 tx=0x7f66e001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526e8acde4fce4b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:13.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66e7fff640 1 -- 172.21.15.12:0/1805304208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00b2070 con 0x7f66f006c140 2024-04-27T07:05:13.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66e7fff640 1 -- 172.21.15.12:0/1805304208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e00a9070 con 0x7f66f006c140 2024-04-27T07:05:13.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66e7fff640 1 -- 172.21.15.12:0/1805304208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00ad880 con 0x7f66f006c140 2024-04-27T07:05:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 -- 172.21.15.12:0/1805304208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f006c140 msgr2=0x7f66f014b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 --2- 172.21.15.12:0/1805304208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f006c140 0x7f66f014b880 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66f00715a0 tx=0x7f66e001c490 comp rx=0 tx=0).stop 2024-04-27T07:05:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 -- 172.21.15.12:0/1805304208 shutdown_connections 2024-04-27T07:05:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 --2- 172.21.15.12:0/1805304208 >> v2:172.21.15.53:3300/0 conn(0x7f66f00b6af0 0x7f66f014bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 --2- 172.21.15.12:0/1805304208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f006c140 0x7f66f014b880 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 --2- 172.21.15.12:0/1805304208 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f00af770 0x7f66f014b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 -- 172.21.15.12:0/1805304208 >> 172.21.15.12:0/1805304208 conn(0x7f66f00b6ff0 msgr2=0x7f66f00b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:13.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 -- 172.21.15.12:0/1805304208 shutdown_connections 2024-04-27T07:05:13.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 -- 172.21.15.12:0/1805304208 wait complete. 2024-04-27T07:05:13.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 Processor -- start 2024-04-27T07:05:13.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 -- start start 2024-04-27T07:05:13.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f006c140 0x7f66f01e1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f00af770 0x7f66f009bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f00b6af0 0x7f66f00b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f66f0148970 con 0x7f66f00af770 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01487f0 con 0x7f66f00b6af0 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f76e6640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f66f0148af0 con 0x7f66f006c140 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f4c81640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f00af770 0x7f66f009bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f4c81640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f00af770 0x7f66f009bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35226/0 (socket says 172.21.15.12:35226) 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f4c81640 1 -- 172.21.15.12:0/3176285764 learned_addr learned my addr 172.21.15.12:0/3176285764 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f5c83640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f00b6af0 0x7f66f00b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f5482640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f006c140 0x7f66f01e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f4c81640 1 -- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f006c140 msgr2=0x7f66f01e1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f4c81640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f006c140 0x7f66f01e1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f4c81640 1 -- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f00b6af0 msgr2=0x7f66f00b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f4c81640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f00b6af0 0x7f66f00b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f4c81640 1 -- 172.21.15.12:0/3176285764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f01e0720 con 0x7f66f00af770 2024-04-27T07:05:13.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f5482640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f006c140 0x7f66f01e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:13.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.532+0000 7f66f5c83640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f00b6af0 0x7f66f00b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:13.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66f4c81640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f00af770 0x7f66f009bd20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f66e001ab40 tx=0x7f66e0019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:13.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66e5ffb640 1 -- 172.21.15.12:0/3176285764 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00b2040 con 0x7f66f00af770 2024-04-27T07:05:13.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66e5ffb640 1 -- 172.21.15.12:0/3176285764 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e00b6620 con 0x7f66f00af770 2024-04-27T07:05:13.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66f76e6640 1 -- 172.21.15.12:0/3176285764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f01e0a00 con 0x7f66f00af770 2024-04-27T07:05:13.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66e5ffb640 1 -- 172.21.15.12:0/3176285764 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e00b6a10 con 0x7f66f00af770 2024-04-27T07:05:13.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66f76e6640 1 -- 172.21.15.12:0/3176285764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f009c620 con 0x7f66f00af770 2024-04-27T07:05:13.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66f76e6640 1 -- 172.21.15.12:0/3176285764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f0069330 con 0x7f66f00af770 2024-04-27T07:05:13.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66e5ffb640 1 -- 172.21.15.12:0/3176285764 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f66e00ab070 con 0x7f66f00af770 2024-04-27T07:05:13.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66e5ffb640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f66d4068380 0x7f66d406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66e5ffb640 1 -- 172.21.15.12:0/3176285764 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f66e0121000 con 0x7f66f00af770 2024-04-27T07:05:13.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66f5482640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f66d4068380 0x7f66d406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.536+0000 7f66f5482640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f66d4068380 0x7f66d406a840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f66e8003fd0 tx=0x7f66e80039e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:13.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.540+0000 7f66e5ffb640 1 -- 172.21.15.12:0/3176285764 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f66e0016110 con 0x7f66f00af770 2024-04-27T07:05:13.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 -- 172.21.15.12:0/3176285764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f66f00b8490 con 0x7f66f00af770 2024-04-27T07:05:13.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66e5ffb640 1 -- 172.21.15.12:0/3176285764 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f66e00f1850 con 0x7f66f00af770 2024-04-27T07:05:13.702 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-27T07:05:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 -- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f66d4068380 msgr2=0x7f66d406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f66d4068380 0x7f66d406a840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f66e8003fd0 tx=0x7f66e80039e0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 -- 172.21.15.12:0/3176285764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f00af770 msgr2=0x7f66f009bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f00af770 0x7f66f009bd20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f66e001ab40 tx=0x7f66e0019230 comp rx=0 tx=0).stop 2024-04-27T07:05:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 -- 172.21.15.12:0/3176285764 shutdown_connections 2024-04-27T07:05:13.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f66d4068380 0x7f66d406a840 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f66f00b6af0 0x7f66f00b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f00af770 0x7f66f009bd20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 --2- 172.21.15.12:0/3176285764 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f66f006c140 0x7f66f01e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 -- 172.21.15.12:0/3176285764 >> 172.21.15.12:0/3176285764 conn(0x7f66f00b6ff0 msgr2=0x7f66f00b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 -- 172.21.15.12:0/3176285764 shutdown_connections 2024-04-27T07:05:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.696+0000 7f66f76e6640 1 -- 172.21.15.12:0/3176285764 wait complete. 2024-04-27T07:05:13.719 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T07:05:13.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.840+0000 7f64711a9640 1 Processor -- start 2024-04-27T07:05:13.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.840+0000 7f64711a9640 1 -- start start 2024-04-27T07:05:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f64711a9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f646c06e370 0x7f646c13faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f64711a9640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f646c0b3650 0x7f646c0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f64711a9640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06d300 0x7f646c0b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f64711a9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f646c0b5c10 con 0x7f646c06e370 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f64711a9640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f646c0b4d70 con 0x7f646c0b3650 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f64711a9640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f646c073e60 con 0x7f646c06d300 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646ad9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f646c06e370 0x7f646c13faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646ad9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f646c06e370 0x7f646c13faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35232/0 (socket says 172.21.15.12:35232) 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646ad9d640 1 -- 172.21.15.12:0/571071828 learned_addr learned my addr 172.21.15.12:0/571071828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646a59c640 1 --2- 172.21.15.12:0/571071828 >> v2:172.21.15.53:3300/0 conn(0x7f646c0b3650 0x7f646c0b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646b59e640 1 --2- 172.21.15.12:0/571071828 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06d300 0x7f646c0b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646ad9d640 1 -- 172.21.15.12:0/571071828 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06d300 msgr2=0x7f646c0b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646ad9d640 1 --2- 172.21.15.12:0/571071828 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06d300 0x7f646c0b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646ad9d640 1 -- 172.21.15.12:0/571071828 >> v2:172.21.15.53:3300/0 conn(0x7f646c0b3650 msgr2=0x7f646c0b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646ad9d640 1 --2- 172.21.15.12:0/571071828 >> v2:172.21.15.53:3300/0 conn(0x7f646c0b3650 0x7f646c0b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646ad9d640 1 -- 172.21.15.12:0/571071828 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c0740d0 con 0x7f646c06e370 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646a59c640 1 --2- 172.21.15.12:0/571071828 >> v2:172.21.15.53:3300/0 conn(0x7f646c0b3650 0x7f646c0b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:13.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646b59e640 1 --2- 172.21.15.12:0/571071828 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06d300 0x7f646c0b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:13.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f646ad9d640 1 --2- 172.21.15.12:0/571071828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c06e370 0x7f646c13faa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f646c068760 tx=0x7f645c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6a899c193a5266 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:13.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f6469d9b640 1 -- 172.21.15.12:0/571071828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c0b0070 con 0x7f646c06e370 2024-04-27T07:05:13.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f6469d9b640 1 -- 172.21.15.12:0/571071828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645c0ab600 con 0x7f646c06e370 2024-04-27T07:05:13.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.844+0000 7f6469d9b640 1 -- 172.21.15.12:0/571071828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c0ab9d0 con 0x7f646c06e370 2024-04-27T07:05:13.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- 172.21.15.12:0/571071828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c06e370 msgr2=0x7f646c13faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 --2- 172.21.15.12:0/571071828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c06e370 0x7f646c13faa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f646c068760 tx=0x7f645c01c490 comp rx=0 tx=0).stop 2024-04-27T07:05:13.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- 172.21.15.12:0/571071828 shutdown_connections 2024-04-27T07:05:13.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 --2- 172.21.15.12:0/571071828 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06d300 0x7f646c0b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 --2- 172.21.15.12:0/571071828 >> v2:172.21.15.53:3300/0 conn(0x7f646c0b3650 0x7f646c0b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 --2- 172.21.15.12:0/571071828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c06e370 0x7f646c13faa0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- 172.21.15.12:0/571071828 >> 172.21.15.12:0/571071828 conn(0x7f646c0b93b0 msgr2=0x7f646c0b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:13.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- 172.21.15.12:0/571071828 shutdown_connections 2024-04-27T07:05:13.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- 172.21.15.12:0/571071828 wait complete. 2024-04-27T07:05:13.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 Processor -- start 2024-04-27T07:05:13.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- start start 2024-04-27T07:05:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f646c06d300 0x7f646c1de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06e370 0x7f646c09d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c0b3650 0x7f646c138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f646c0b67d0 con 0x7f646c0b3650 2024-04-27T07:05:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f646c0b6650 con 0x7f646c06d300 2024-04-27T07:05:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f646c0b6950 con 0x7f646c06e370 2024-04-27T07:05:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646b59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c0b3650 0x7f646c138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646b59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c0b3650 0x7f646c138100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35248/0 (socket says 172.21.15.12:35248) 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646b59e640 1 -- 172.21.15.12:0/2177488485 learned_addr learned my addr 172.21.15.12:0/2177488485 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646ad9d640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f646c06d300 0x7f646c1de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646a59c640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06e370 0x7f646c09d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646b59e640 1 -- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06e370 msgr2=0x7f646c09d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646b59e640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06e370 0x7f646c09d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646b59e640 1 -- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f646c06d300 msgr2=0x7f646c1de1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646b59e640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f646c06d300 0x7f646c1de1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646b59e640 1 -- 172.21.15.12:0/2177488485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c1e1850 con 0x7f646c0b3650 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646a59c640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06e370 0x7f646c09d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f646b59e640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c0b3650 0x7f646c138100 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f646400ce70 tx=0x7f646400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64537fe640 1 -- 172.21.15.12:0/2177488485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464003ba0 con 0x7f646c0b3650 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64537fe640 1 -- 172.21.15.12:0/2177488485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6464003d60 con 0x7f646c0b3650 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- 172.21.15.12:0/2177488485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646c1e1b30 con 0x7f646c0b3650 2024-04-27T07:05:13.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64537fe640 1 -- 172.21.15.12:0/2177488485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646400d880 con 0x7f646c0b3650 2024-04-27T07:05:13.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.848+0000 7f64711a9640 1 -- 172.21.15.12:0/2177488485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646c1de6f0 con 0x7f646c0b3650 2024-04-27T07:05:13.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.852+0000 7f64711a9640 1 -- 172.21.15.12:0/2177488485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f646c068c20 con 0x7f646c0b3650 2024-04-27T07:05:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.852+0000 7f64537fe640 1 -- 172.21.15.12:0/2177488485 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f646400da20 con 0x7f646c0b3650 2024-04-27T07:05:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.852+0000 7f64537fe640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64580683b0 0x7f645806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.852+0000 7f64537fe640 1 -- 172.21.15.12:0/2177488485 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f6464087290 con 0x7f646c0b3650 2024-04-27T07:05:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.852+0000 7f646ad9d640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64580683b0 0x7f645806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.852+0000 7f646ad9d640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64580683b0 0x7f645806a870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f646c069d80 tx=0x7f645c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:13.856+0000 7f64537fe640 1 -- 172.21.15.12:0/2177488485 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f64640578e0 con 0x7f646c0b3650 2024-04-27T07:05:14.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 -- 172.21.15.12:0/2177488485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f646c1e2050 con 0x7f646c0b3650 2024-04-27T07:05:14.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64537fe640 1 -- 172.21.15.12:0/2177488485 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f64640571e0 con 0x7f646c0b3650 2024-04-27T07:05:14.022 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-27T07:05:14.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 -- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64580683b0 msgr2=0x7f645806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64580683b0 0x7f645806a870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f646c069d80 tx=0x7f645c008040 comp rx=0 tx=0).stop 2024-04-27T07:05:14.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 -- 172.21.15.12:0/2177488485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c0b3650 msgr2=0x7f646c138100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c0b3650 0x7f646c138100 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f646400ce70 tx=0x7f646400a710 comp rx=0 tx=0).stop 2024-04-27T07:05:14.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 -- 172.21.15.12:0/2177488485 shutdown_connections 2024-04-27T07:05:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f64580683b0 0x7f645806a870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f646c0b3650 0x7f646c138100 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f646c06e370 0x7f646c09d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 --2- 172.21.15.12:0/2177488485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f646c06d300 0x7f646c1de1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 -- 172.21.15.12:0/2177488485 >> 172.21.15.12:0/2177488485 conn(0x7f646c0b93b0 msgr2=0x7f646c0af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 -- 172.21.15.12:0/2177488485 shutdown_connections 2024-04-27T07:05:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.016+0000 7f64711a9640 1 -- 172.21.15.12:0/2177488485 wait complete. 2024-04-27T07:05:14.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:05:14.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.152+0000 7f3a6d9e6640 1 Processor -- start 2024-04-27T07:05:14.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.152+0000 7f3a6d9e6640 1 -- start start 2024-04-27T07:05:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a680baa00 0x7f3a6814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3a68141620 0x7f3a68141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a680b7990 con 0x7f3a68147c00 2024-04-27T07:05:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f3a680b73d0 con 0x7f3a68141620 2024-04-27T07:05:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3a680bade0 con 0x7f3a680baa00 2024-04-27T07:05:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a677fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3a68141620 0x7f3a68141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a677fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3a68141620 0x7f3a68141ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35264/0 (socket says 172.21.15.12:35264) 2024-04-27T07:05:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a677fe640 1 -- 172.21.15.12:0/2714786762 learned_addr learned my addr 172.21.15.12:0/2714786762 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a66ffd640 1 --2- 172.21.15.12:0/2714786762 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a680baa00 0x7f3a6814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a677fe640 1 -- 172.21.15.12:0/2714786762 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a680baa00 msgr2=0x7f3a6814b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:14.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a677fe640 1 --2- 172.21.15.12:0/2714786762 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a680baa00 0x7f3a6814b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a677fe640 1 -- 172.21.15.12:0/2714786762 >> v1:172.21.15.53:6789/0 conn(0x7f3a68147c00 legacy=0x7f3a68148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a677fe640 1 -- 172.21.15.12:0/2714786762 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a680b76b0 con 0x7f3a68141620 2024-04-27T07:05:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a677fe640 1 --2- 172.21.15.12:0/2714786762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68141620 0x7f3a68141ab0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009e50 tx=0x7f3a6001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27b18d6754e6a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a65ffb640 1 -- 172.21.15.12:0/2714786762 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a600a57a0 con 0x7f3a68141620 2024-04-27T07:05:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a65ffb640 1 -- 172.21.15.12:0/2714786762 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a600a5940 con 0x7f3a68141620 2024-04-27T07:05:14.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a65ffb640 1 -- 172.21.15.12:0/2714786762 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a600a5d10 con 0x7f3a68141620 2024-04-27T07:05:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/2714786762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68141620 msgr2=0x7f3a68141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 --2- 172.21.15.12:0/2714786762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68141620 0x7f3a68141ab0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009e50 tx=0x7f3a6001c8e0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/2714786762 shutdown_connections 2024-04-27T07:05:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 --2- 172.21.15.12:0/2714786762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68141620 0x7f3a68141ab0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 --2- 172.21.15.12:0/2714786762 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a680baa00 0x7f3a6814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/2714786762 >> 172.21.15.12:0/2714786762 conn(0x7f3a680738e0 msgr2=0x7f3a680ba090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/2714786762 shutdown_connections 2024-04-27T07:05:14.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.156+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/2714786762 wait complete. 2024-04-27T07:05:14.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 Processor -- start 2024-04-27T07:05:14.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 -- start start 2024-04-27T07:05:14.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a680baa00 0x7f3a6813cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a68141620 0x7f3a680bc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68147c00 0x7f3a681346e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68154da0 con 0x7f3a68147c00 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68141ff0 con 0x7f3a68141620 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3a68154f20 con 0x7f3a680baa00 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68147c00 0x7f3a681346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68147c00 0x7f3a681346e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35268/0 (socket says 172.21.15.12:35268) 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a677fe640 1 -- 172.21.15.12:0/1460637734 learned_addr learned my addr 172.21.15.12:0/1460637734 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a667fc640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a68141620 0x7f3a680bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a66ffd640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a680baa00 0x7f3a6813cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a677fe640 1 -- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a680baa00 msgr2=0x7f3a6813cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a677fe640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a680baa00 0x7f3a6813cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a677fe640 1 -- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a68141620 msgr2=0x7f3a680bc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a677fe640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a68141620 0x7f3a680bc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a677fe640 1 -- 172.21.15.12:0/1460637734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6813aaa0 con 0x7f3a68147c00 2024-04-27T07:05:14.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a667fc640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a68141620 0x7f3a680bc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:14.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a677fe640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68147c00 0x7f3a681346e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3a6001a510 tx=0x7f3a60019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:14.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a4ffff640 1 -- 172.21.15.12:0/1460637734 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a600bc020 con 0x7f3a68147c00 2024-04-27T07:05:14.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a4ffff640 1 -- 172.21.15.12:0/1460637734 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a600b8670 con 0x7f3a68147c00 2024-04-27T07:05:14.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/1460637734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6813c760 con 0x7f3a68147c00 2024-04-27T07:05:14.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a4ffff640 1 -- 172.21.15.12:0/1460637734 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a600b8a60 con 0x7f3a68147c00 2024-04-27T07:05:14.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/1460637734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a680bca80 con 0x7f3a68147c00 2024-04-27T07:05:14.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.160+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/1460637734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a68068c20 con 0x7f3a68147c00 2024-04-27T07:05:14.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.164+0000 7f3a4ffff640 1 -- 172.21.15.12:0/1460637734 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3a600bf020 con 0x7f3a68147c00 2024-04-27T07:05:14.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.164+0000 7f3a4ffff640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3a540683b0 0x7f3a5406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.164+0000 7f3a4ffff640 1 -- 172.21.15.12:0/1460637734 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f3a60121c10 con 0x7f3a68147c00 2024-04-27T07:05:14.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.164+0000 7f3a66ffd640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3a540683b0 0x7f3a5406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.164+0000 7f3a66ffd640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3a540683b0 0x7f3a5406a870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800b9d0 tx=0x7f3a58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:14.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.168+0000 7f3a4ffff640 1 -- 172.21.15.12:0/1460637734 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3a600c6030 con 0x7f3a68147c00 2024-04-27T07:05:14.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.364+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/1460637734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a68146470 con 0x7f3a68147c00 2024-04-27T07:05:14.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.364+0000 7f3a4ffff640 1 -- 172.21.15.12:0/1460637734 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3531 (secure 0 0 0) 0x7f3a60016620 con 0x7f3a68147c00 2024-04-27T07:05:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:14.371 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3153112684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3153112684},{"type":"v1","addr":"172.21.15.53:6835","nonce":3153112684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:05:00.918068+0000","modified":"2024-04-27T07:05:03.947324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5001":{"gid":5001,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4067812811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4067812811},{"type":"v1","addr":"172.21.15.12:6833","nonce":4067812811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5001,"qdb_cluster":[5001]},"id":2}]} 2024-04-27T07:05:14.371 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 16 2024-04-27T07:05:14.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3a540683b0 msgr2=0x7f3a5406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3a540683b0 0x7f3a5406a870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800b9d0 tx=0x7f3a58008040 comp rx=0 tx=0).stop 2024-04-27T07:05:14.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/1460637734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68147c00 msgr2=0x7f3a681346e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68147c00 0x7f3a681346e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3a6001a510 tx=0x7f3a60019680 comp rx=0 tx=0).stop 2024-04-27T07:05:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/1460637734 shutdown_connections 2024-04-27T07:05:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3a540683b0 0x7f3a5406a870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3a68147c00 0x7f3a681346e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a68141620 0x7f3a680bc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 --2- 172.21.15.12:0/1460637734 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3a680baa00 0x7f3a6813cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/1460637734 >> 172.21.15.12:0/1460637734 conn(0x7f3a680738e0 msgr2=0x7f3a68145b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/1460637734 shutdown_connections 2024-04-27T07:05:14.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.368+0000 7f3a6d9e6640 1 -- 172.21.15.12:0/1460637734 wait complete. 2024-04-27T07:05:14.390 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:05:14.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8f3fc640 1 Processor -- start 2024-04-27T07:05:14.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8f3fc640 1 -- start start 2024-04-27T07:05:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8f3fc640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f880b8b10 0x7f4f88146450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8f3fc640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4f88146990 0x7f4f8813f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8f3fc640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f8806e520 con 0x7f4f880b8730 2024-04-27T07:05:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8f3fc640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f4f88145c70 con 0x7f4f88146990 2024-04-27T07:05:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8f3fc640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f4f880b8f70 con 0x7f4f880b8b10 2024-04-27T07:05:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8d999640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4f88146990 0x7f4f8813f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8c997640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f880b8b10 0x7f4f88146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8d999640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f4f88146990 0x7f4f8813f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:54260/0 (socket says 172.21.15.12:54260) 2024-04-27T07:05:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8d198640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7f4f880b8730 0x7f4f881496f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:58788/0 (socket says 172.21.15.12:58788) 2024-04-27T07:05:14.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8d999640 1 -- 172.21.15.12:0/1095548711 learned_addr learned my addr 172.21.15.12:0/1095548711 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8d999640 1 -- 172.21.15.12:0/1095548711 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f880b8b10 msgr2=0x7f4f88146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8d999640 1 --2- 172.21.15.12:0/1095548711 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f880b8b10 0x7f4f88146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8d999640 1 -- 172.21.15.12:0/1095548711 >> v1:172.21.15.53:6789/0 conn(0x7f4f880b8730 legacy=0x7f4f881496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.504+0000 7f4f8d999640 1 -- 172.21.15.12:0/1095548711 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f88149fb0 con 0x7f4f88146990 2024-04-27T07:05:14.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d999640 1 --2- 172.21.15.12:0/1095548711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f88146990 0x7f4f8813f940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009e50 tx=0x7f4f8001c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea88624a95bc4bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:14.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f7ffff640 1 -- 172.21.15.12:0/1095548711 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f800a56e0 con 0x7f4f88146990 2024-04-27T07:05:14.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f7ffff640 1 -- 172.21.15.12:0/1095548711 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f800a5880 con 0x7f4f88146990 2024-04-27T07:05:14.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f7ffff640 1 -- 172.21.15.12:0/1095548711 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f800a5c50 con 0x7f4f88146990 2024-04-27T07:05:14.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/1095548711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f88146990 msgr2=0x7f4f8813f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 --2- 172.21.15.12:0/1095548711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f88146990 0x7f4f8813f940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009e50 tx=0x7f4f8001c910 comp rx=0 tx=0).stop 2024-04-27T07:05:14.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/1095548711 shutdown_connections 2024-04-27T07:05:14.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 --2- 172.21.15.12:0/1095548711 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f88146990 0x7f4f8813f940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 --2- 172.21.15.12:0/1095548711 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f880b8b10 0x7f4f88146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/1095548711 >> 172.21.15.12:0/1095548711 conn(0x7f4f8809c480 msgr2=0x7f4f8809c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:14.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/1095548711 shutdown_connections 2024-04-27T07:05:14.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/1095548711 wait complete. 2024-04-27T07:05:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 Processor -- start 2024-04-27T07:05:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 -- start start 2024-04-27T07:05:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f880b8730 0x7f4f88148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f880b8b10 0x7f4f880afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f88146990 0x7f4f880b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88153180 con 0x7f4f88146990 2024-04-27T07:05:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88153000 con 0x7f4f880b8b10 2024-04-27T07:05:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8f3fc640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f4f88153300 con 0x7f4f880b8730 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d999640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f88146990 0x7f4f880b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d999640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f88146990 0x7f4f880b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35280/0 (socket says 172.21.15.12:35280) 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d999640 1 -- 172.21.15.12:0/2099126040 learned_addr learned my addr 172.21.15.12:0/2099126040 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d198640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f880b8730 0x7f4f88148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8c997640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f880b8b10 0x7f4f880afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d999640 1 -- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f880b8730 msgr2=0x7f4f88148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d999640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f880b8730 0x7f4f88148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d999640 1 -- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f880b8b10 msgr2=0x7f4f880afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d999640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f880b8b10 0x7f4f880afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d999640 1 -- 172.21.15.12:0/2099126040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f88147c50 con 0x7f4f88146990 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8c997640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f880b8b10 0x7f4f880afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.508+0000 7f4f8d999640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f88146990 0x7f4f880b0a80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4f8001a540 tx=0x7f4f800196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f7dffb640 1 -- 172.21.15.12:0/2099126040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f800bc020 con 0x7f4f88146990 2024-04-27T07:05:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f7dffb640 1 -- 172.21.15.12:0/2099126040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f800b8880 con 0x7f4f88146990 2024-04-27T07:05:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/2099126040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f881489e0 con 0x7f4f88146990 2024-04-27T07:05:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f7dffb640 1 -- 172.21.15.12:0/2099126040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80009510 con 0x7f4f88146990 2024-04-27T07:05:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/2099126040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f88147ec0 con 0x7f4f88146990 2024-04-27T07:05:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/2099126040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f88069330 con 0x7f4f88146990 2024-04-27T07:05:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f7dffb640 1 -- 172.21.15.12:0/2099126040 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4f800bf020 con 0x7f4f88146990 2024-04-27T07:05:14.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f7dffb640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4f6c068380 0x7f4f6c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f7dffb640 1 -- 172.21.15.12:0/2099126040 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f4f80121a70 con 0x7f4f88146990 2024-04-27T07:05:14.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f8d198640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4f6c068380 0x7f4f6c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.512+0000 7f4f8d198640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4f6c068380 0x7f4f6c06a840 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f84000f70 tx=0x7f4f84007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:14.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.516+0000 7f4f7dffb640 1 -- 172.21.15.12:0/2099126040 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4f800c6030 con 0x7f4f88146990 2024-04-27T07:05:14.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.672+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/2099126040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f880a07b0 con 0x7f4f88146990 2024-04-27T07:05:14.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.676+0000 7f4f7dffb640 1 -- 172.21.15.12:0/2099126040 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16609 (secure 0 0 0) 0x7f4f80016620 con 0x7f4f88146990 2024-04-27T07:05:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:14.681 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:05:02.565909+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:59.279361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:59.893681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:05:14.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4f6c068380 msgr2=0x7f4f6c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4f6c068380 0x7f4f6c06a840 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f84000f70 tx=0x7f4f84007c90 comp rx=0 tx=0).stop 2024-04-27T07:05:14.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/2099126040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f88146990 msgr2=0x7f4f880b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f88146990 0x7f4f880b0a80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4f8001a540 tx=0x7f4f800196b0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/2099126040 shutdown_connections 2024-04-27T07:05:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f4f6c068380 0x7f4f6c06a840 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f88146990 0x7f4f880b0a80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f880b8b10 0x7f4f880afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 --2- 172.21.15.12:0/2099126040 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f4f880b8730 0x7f4f88148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/2099126040 >> 172.21.15.12:0/2099126040 conn(0x7f4f8809c480 msgr2=0x7f4f8809c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/2099126040 shutdown_connections 2024-04-27T07:05:14.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.680+0000 7f4f8f3fc640 1 -- 172.21.15.12:0/2099126040 wait complete. 2024-04-27T07:05:14.701 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-27T07:05:14.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:05:14.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.812+0000 7f0fcc401640 1 Processor -- start 2024-04-27T07:05:14.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.812+0000 7f0fcc401640 1 -- start start 2024-04-27T07:05:14.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0fc40b9a10 0x7f0fc40b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f0fc414e400 0x7f0fc41507f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc406dac0 con 0x7f0fc40b3680 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc406e890 con 0x7f0fc40b9a10 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc40b9df0 con 0x7f0fc414e400 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fca19d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0fc40b9a10 0x7f0fc40b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fca19d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0fc40b9a10 0x7f0fc40b65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35296/0 (socket says 172.21.15.12:35296) 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fca19d640 1 -- 172.21.15.12:0/1321556381 learned_addr learned my addr 172.21.15.12:0/1321556381 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fca99e640 1 --2- 172.21.15.12:0/1321556381 >> v2:172.21.15.53:3300/0 conn(0x7f0fc414e400 0x7f0fc41507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fca19d640 1 -- 172.21.15.12:0/1321556381 >> v2:172.21.15.53:3300/0 conn(0x7f0fc414e400 msgr2=0x7f0fc41507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fca19d640 1 --2- 172.21.15.12:0/1321556381 >> v2:172.21.15.53:3300/0 conn(0x7f0fc414e400 0x7f0fc41507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fca19d640 1 -- 172.21.15.12:0/1321556381 >> v1:172.21.15.53:6789/0 conn(0x7f0fc40b3680 legacy=0x7f0fc4145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fca19d640 1 -- 172.21.15.12:0/1321556381 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc406e090 con 0x7f0fc40b9a10 2024-04-27T07:05:14.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fca99e640 1 --2- 172.21.15.12:0/1321556381 >> v2:172.21.15.53:3300/0 conn(0x7f0fc414e400 0x7f0fc41507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:14.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fca19d640 1 --2- 172.21.15.12:0/1321556381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b9a10 0x7f0fc40b65c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0fc406edf0 tx=0x7f0fc001c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843ded546a5ae978 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:14.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fc919b640 1 -- 172.21.15.12:0/1321556381 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00b0070 con 0x7f0fc40b9a10 2024-04-27T07:05:14.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fc919b640 1 -- 172.21.15.12:0/1321556381 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc00ab600 con 0x7f0fc40b9a10 2024-04-27T07:05:14.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fc919b640 1 -- 172.21.15.12:0/1321556381 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00ab9d0 con 0x7f0fc40b9a10 2024-04-27T07:05:14.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 -- 172.21.15.12:0/1321556381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b9a10 msgr2=0x7f0fc40b65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 --2- 172.21.15.12:0/1321556381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b9a10 0x7f0fc40b65c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0fc406edf0 tx=0x7f0fc001c270 comp rx=0 tx=0).stop 2024-04-27T07:05:14.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 -- 172.21.15.12:0/1321556381 shutdown_connections 2024-04-27T07:05:14.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 --2- 172.21.15.12:0/1321556381 >> v2:172.21.15.53:3300/0 conn(0x7f0fc414e400 0x7f0fc41507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 --2- 172.21.15.12:0/1321556381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b9a10 0x7f0fc40b65c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.816+0000 7f0fcc401640 1 -- 172.21.15.12:0/1321556381 >> 172.21.15.12:0/1321556381 conn(0x7f0fc4067d10 msgr2=0x7f0fc4068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:14.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 -- 172.21.15.12:0/1321556381 shutdown_connections 2024-04-27T07:05:14.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 -- 172.21.15.12:0/1321556381 wait complete. 2024-04-27T07:05:14.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 Processor -- start 2024-04-27T07:05:14.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 -- start start 2024-04-27T07:05:14.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b3680 0x7f0fc414c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fc40b9a10 0x7f0fc414bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fc414e400 0x7f0fc41414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4074020 con 0x7f0fc40b3680 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4073ea0 con 0x7f0fc40b9a10 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc40b6b00 con 0x7f0fc414e400 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fca19d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b3680 0x7f0fc414c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fca19d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b3680 0x7f0fc414c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35308/0 (socket says 172.21.15.12:35308) 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fca19d640 1 -- 172.21.15.12:0/3715849487 learned_addr learned my addr 172.21.15.12:0/3715849487 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fc999c640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fc40b9a10 0x7f0fc414bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fca99e640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fc414e400 0x7f0fc41414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fca19d640 1 -- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fc414e400 msgr2=0x7f0fc41414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fca19d640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fc414e400 0x7f0fc41414e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fca19d640 1 -- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fc40b9a10 msgr2=0x7f0fc414bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fca19d640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fc40b9a10 0x7f0fc414bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fca19d640 1 -- 172.21.15.12:0/3715849487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc414cfb0 con 0x7f0fc40b3680 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fc999c640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fc40b9a10 0x7f0fc414bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fca19d640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b3680 0x7f0fc414c880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00ae040 tx=0x7f0fc0019010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:14.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fb6ffd640 1 -- 172.21.15.12:0/3715849487 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00b8020 con 0x7f0fc40b3680 2024-04-27T07:05:14.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fb6ffd640 1 -- 172.21.15.12:0/3715849487 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc00b4610 con 0x7f0fc40b3680 2024-04-27T07:05:14.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fb6ffd640 1 -- 172.21.15.12:0/3715849487 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00b0040 con 0x7f0fc40b3680 2024-04-27T07:05:14.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 -- 172.21.15.12:0/3715849487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc414b2e0 con 0x7f0fc40b3680 2024-04-27T07:05:14.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.820+0000 7f0fcc401640 1 -- 172.21.15.12:0/3715849487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc414df00 con 0x7f0fc40b3680 2024-04-27T07:05:14.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.824+0000 7f0fcc401640 1 -- 172.21.15.12:0/3715849487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc4068c20 con 0x7f0fc40b3680 2024-04-27T07:05:14.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.824+0000 7f0fb6ffd640 1 -- 172.21.15.12:0/3715849487 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0fc00bb020 con 0x7f0fc40b3680 2024-04-27T07:05:14.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.824+0000 7f0fb6ffd640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fa8068380 0x7f0fa806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:14.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.824+0000 7f0fb6ffd640 1 -- 172.21.15.12:0/3715849487 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f0fc01207e0 con 0x7f0fc40b3680 2024-04-27T07:05:14.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.824+0000 7f0fc999c640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fa8068380 0x7f0fa806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:14.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.824+0000 7f0fc999c640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fa8068380 0x7f0fa806a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0fc414b190 tx=0x7f0fb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:14.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:14.828+0000 7f0fb6ffd640 1 -- 172.21.15.12:0/3715849487 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0fc00c2030 con 0x7f0fc40b3680 2024-04-27T07:05:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.028+0000 7f0fcc401640 1 -- 172.21.15.12:0/3715849487 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fc406d7f0 con 0x7f0fc40b3680 2024-04-27T07:05:15.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.028+0000 7f0fb6ffd640 1 -- 172.21.15.12:0/3715849487 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3531 (secure 0 0 0) 0x7f0fc406dac0 con 0x7f0fc40b3680 2024-04-27T07:05:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:15.035 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3153112684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3153112684},{"type":"v1","addr":"172.21.15.53:6835","nonce":3153112684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:05:00.918068+0000","modified":"2024-04-27T07:05:03.947324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5001":{"gid":5001,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4067812811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4067812811},{"type":"v1","addr":"172.21.15.12:6833","nonce":4067812811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5001,"qdb_cluster":[5001]},"id":2}]} 2024-04-27T07:05:15.035 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 16 2024-04-27T07:05:15.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 -- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fa8068380 msgr2=0x7f0fa806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fa8068380 0x7f0fa806a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0fc414b190 tx=0x7f0fb8009040 comp rx=0 tx=0).stop 2024-04-27T07:05:15.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 -- 172.21.15.12:0/3715849487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b3680 msgr2=0x7f0fc414c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b3680 0x7f0fc414c880 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00ae040 tx=0x7f0fc0019010 comp rx=0 tx=0).stop 2024-04-27T07:05:15.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 -- 172.21.15.12:0/3715849487 shutdown_connections 2024-04-27T07:05:15.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0fa8068380 0x7f0fa806a840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0fc414e400 0x7f0fc41414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fc40b9a10 0x7f0fc414bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 --2- 172.21.15.12:0/3715849487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fc40b3680 0x7f0fc414c880 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 -- 172.21.15.12:0/3715849487 >> 172.21.15.12:0/3715849487 conn(0x7f0fc4067d10 msgr2=0x7f0fc409b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 -- 172.21.15.12:0/3715849487 shutdown_connections 2024-04-27T07:05:15.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.032+0000 7f0fcc401640 1 -- 172.21.15.12:0/3715849487 wait complete. 2024-04-27T07:05:15.052 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:05:15.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56cef69640 1 Processor -- start 2024-04-27T07:05:15.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56cef69640 1 -- start start 2024-04-27T07:05:15.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56cef69640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f56c80b4500 0x7f56c80b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56cef69640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56c813f7e0 0x7f56c813fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56cef69640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f56c8145ec0 0x7f56c8146350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56cef69640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f56c8140180 con 0x7f56c80b4500 2024-04-27T07:05:15.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56cef69640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f56c8146890 con 0x7f56c8145ec0 2024-04-27T07:05:15.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56cef69640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f56c8145a00 con 0x7f56c813f7e0 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56ccd05640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f56c80b4500 0x7f56c80b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56ccd05640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f56c80b4500 0x7f56c80b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35320/0 (socket says 172.21.15.12:35320) 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56ccd05640 1 -- 172.21.15.12:0/2950176716 learned_addr learned my addr 172.21.15.12:0/2950176716 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56cd506640 1 --2- 172.21.15.12:0/2950176716 >> v2:172.21.15.53:3300/0 conn(0x7f56c8145ec0 0x7f56c8146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56c7fff640 1 --2- 172.21.15.12:0/2950176716 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56c813f7e0 0x7f56c813fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56ccd05640 1 -- 172.21.15.12:0/2950176716 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56c813f7e0 msgr2=0x7f56c813fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56ccd05640 1 --2- 172.21.15.12:0/2950176716 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56c813f7e0 0x7f56c813fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56ccd05640 1 -- 172.21.15.12:0/2950176716 >> v2:172.21.15.53:3300/0 conn(0x7f56c8145ec0 msgr2=0x7f56c8146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56ccd05640 1 --2- 172.21.15.12:0/2950176716 >> v2:172.21.15.53:3300/0 conn(0x7f56c8145ec0 0x7f56c8146350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56ccd05640 1 -- 172.21.15.12:0/2950176716 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c80b52b0 con 0x7f56c80b4500 2024-04-27T07:05:15.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.172+0000 7f56cd506640 1 --2- 172.21.15.12:0/2950176716 >> v2:172.21.15.53:3300/0 conn(0x7f56c8145ec0 0x7f56c8146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:15.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56ccd05640 1 --2- 172.21.15.12:0/2950176716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c80b4500 0x7f56c80b48e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f56c806c9d0 tx=0x7f56b801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e45fcd8909f8907 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:15.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56c77fe640 1 -- 172.21.15.12:0/2950176716 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b80a5670 con 0x7f56c80b4500 2024-04-27T07:05:15.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56c77fe640 1 -- 172.21.15.12:0/2950176716 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b80a5810 con 0x7f56c80b4500 2024-04-27T07:05:15.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56c77fe640 1 -- 172.21.15.12:0/2950176716 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b80a5be0 con 0x7f56c80b4500 2024-04-27T07:05:15.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56cef69640 1 -- 172.21.15.12:0/2950176716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c80b4500 msgr2=0x7f56c80b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56cef69640 1 --2- 172.21.15.12:0/2950176716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c80b4500 0x7f56c80b48e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f56c806c9d0 tx=0x7f56b801c9f0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56cef69640 1 -- 172.21.15.12:0/2950176716 shutdown_connections 2024-04-27T07:05:15.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56cef69640 1 --2- 172.21.15.12:0/2950176716 >> v2:172.21.15.53:3300/0 conn(0x7f56c8145ec0 0x7f56c8146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56cef69640 1 --2- 172.21.15.12:0/2950176716 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56c813f7e0 0x7f56c813fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56cef69640 1 --2- 172.21.15.12:0/2950176716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c80b4500 0x7f56c80b48e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56cef69640 1 -- 172.21.15.12:0/2950176716 >> 172.21.15.12:0/2950176716 conn(0x7f56c80bb4f0 msgr2=0x7f56c80bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:15.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56cef69640 1 -- 172.21.15.12:0/2950176716 shutdown_connections 2024-04-27T07:05:15.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56cef69640 1 -- 172.21.15.12:0/2950176716 wait complete. 2024-04-27T07:05:15.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.176+0000 7f56cef69640 1 Processor -- start 2024-04-27T07:05:15.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cef69640 1 -- start start 2024-04-27T07:05:15.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cef69640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56c80b4500 0x7f56c81386f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cef69640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c813f7e0 0x7f56c806e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cef69640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c8145ec0 0x7f56c814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cef69640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8149b90 con 0x7f56c813f7e0 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cef69640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f56c8149a10 con 0x7f56c8145ec0 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cef69640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f56c8149d10 con 0x7f56c80b4500 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c813f7e0 0x7f56c806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56ccd05640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56c80b4500 0x7f56c81386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c813f7e0 0x7f56c806e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35326/0 (socket says 172.21.15.12:35326) 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cd506640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c8145ec0 0x7f56c814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c7fff640 1 -- 172.21.15.12:0/1484738432 learned_addr learned my addr 172.21.15.12:0/1484738432 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c7fff640 1 -- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56c80b4500 msgr2=0x7f56c81386f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c7fff640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56c80b4500 0x7f56c81386f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c7fff640 1 -- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c8145ec0 msgr2=0x7f56c814f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c7fff640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c8145ec0 0x7f56c814f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c7fff640 1 -- 172.21.15.12:0/1484738432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c8140440 con 0x7f56c813f7e0 2024-04-27T07:05:15.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c7fff640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c813f7e0 0x7f56c806e520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00d800 tx=0x7f56bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:15.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c57fa640 1 -- 172.21.15.12:0/1484738432 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc018020 con 0x7f56c813f7e0 2024-04-27T07:05:15.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c57fa640 1 -- 172.21.15.12:0/1484738432 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56bc0096a0 con 0x7f56c813f7e0 2024-04-27T07:05:15.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cef69640 1 -- 172.21.15.12:0/1484738432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c806d5c0 con 0x7f56c813f7e0 2024-04-27T07:05:15.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c57fa640 1 -- 172.21.15.12:0/1484738432 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc009a50 con 0x7f56c813f7e0 2024-04-27T07:05:15.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cef69640 1 -- 172.21.15.12:0/1484738432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c814fcf0 con 0x7f56c813f7e0 2024-04-27T07:05:15.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56cef69640 1 -- 172.21.15.12:0/1484738432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c806d990 con 0x7f56c813f7e0 2024-04-27T07:05:15.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c57fa640 1 -- 172.21.15.12:0/1484738432 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f56bc01b020 con 0x7f56c813f7e0 2024-04-27T07:05:15.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.180+0000 7f56c57fa640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f56b4068380 0x7f56b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.184+0000 7f56c57fa640 1 -- 172.21.15.12:0/1484738432 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f56bc086cf0 con 0x7f56c813f7e0 2024-04-27T07:05:15.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.184+0000 7f56ccd05640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f56b4068380 0x7f56b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.184+0000 7f56ccd05640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f56b4068380 0x7f56b406a840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f56b80ae000 tx=0x7f56b80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:15.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.188+0000 7f56c57fa640 1 -- 172.21.15.12:0/1484738432 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f56bc08b070 con 0x7f56c813f7e0 2024-04-27T07:05:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.376+0000 7f56cef69640 1 -- 172.21.15.12:0/1484738432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56c806d790 con 0x7f56c813f7e0 2024-04-27T07:05:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56c57fa640 1 -- 172.21.15.12:0/1484738432 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3531 (secure 0 0 0) 0x7f56c806d790 con 0x7f56c813f7e0 2024-04-27T07:05:15.385 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:15.385 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3153112684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3153112684},{"type":"v1","addr":"172.21.15.53:6835","nonce":3153112684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-27T07:05:00.918068+0000","modified":"2024-04-27T07:05:03.947324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5001":{"gid":5001,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.12:6833/4067812811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4067812811},{"type":"v1","addr":"172.21.15.12:6833","nonce":4067812811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5001,"qdb_cluster":[5001]},"id":2}]} 2024-04-27T07:05:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 16 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 -- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f56b4068380 msgr2=0x7f56b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f56b4068380 0x7f56b406a840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f56b80ae000 tx=0x7f56b80a7070 comp rx=0 tx=0).stop 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 -- 172.21.15.12:0/1484738432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c813f7e0 msgr2=0x7f56c806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c813f7e0 0x7f56c806e520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00d800 tx=0x7f56bc00dcc0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 -- 172.21.15.12:0/1484738432 shutdown_connections 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f56b4068380 0x7f56b406a840 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f56c8145ec0 0x7f56c814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56c813f7e0 0x7f56c806e520 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 --2- 172.21.15.12:0/1484738432 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f56c80b4500 0x7f56c81386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 -- 172.21.15.12:0/1484738432 >> 172.21.15.12:0/1484738432 conn(0x7f56c80bb4f0 msgr2=0x7f56c80b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 -- 172.21.15.12:0/1484738432 shutdown_connections 2024-04-27T07:05:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.380+0000 7f56cef69640 1 -- 172.21.15.12:0/1484738432 wait complete. 2024-04-27T07:05:15.402 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:05:15.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61bb04640 1 Processor -- start 2024-04-27T07:05:15.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61bb04640 1 -- start start 2024-04-27T07:05:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61bb04640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6140b5ae0 0x7fb6140b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61bb04640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6140b7560 con 0x7fb614148810 2024-04-27T07:05:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61bb04640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6140b68a0 con 0x7fb6140b64c0 2024-04-27T07:05:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61bb04640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fb614148c80 con 0x7fb6140b5ae0 2024-04-27T07:05:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61909f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6140b5ae0 0x7fb6140b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61909f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6140b5ae0 0x7fb6140b7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35328/0 (socket says 172.21.15.12:35328) 2024-04-27T07:05:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61909f640 1 -- 172.21.15.12:0/3641714773 learned_addr learned my addr 172.21.15.12:0/3641714773 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61909f640 1 -- 172.21.15.12:0/3641714773 >> v1:172.21.15.53:6789/0 conn(0x7fb6140b64c0 legacy=0x7fb614141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:15.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61909f640 1 -- 172.21.15.12:0/3641714773 >> v1:172.21.15.12:6789/0 conn(0x7fb614148810 legacy=0x7fb614152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61889e640 1 -- 172.21.15.12:0/3641714773 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2963669311 0 0) 0x7fb6140b7560 con 0x7fb614148810 2024-04-27T07:05:15.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61909f640 1 -- 172.21.15.12:0/3641714773 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb614148010 con 0x7fb6140b5ae0 2024-04-27T07:05:15.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61909f640 1 --2- 172.21.15.12:0/3641714773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 0x7fb6140b7f60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb604009870 tx=0x7fb60401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b26304b23c25ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:15.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61889e640 1 -- 172.21.15.12:0/3641714773 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6040b1070 con 0x7fb6140b5ae0 2024-04-27T07:05:15.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.508+0000 7fb61889e640 1 -- 172.21.15.12:0/3641714773 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6040ac640 con 0x7fb6140b5ae0 2024-04-27T07:05:15.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61889e640 1 -- 172.21.15.12:0/3641714773 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6040aca10 con 0x7fb6140b5ae0 2024-04-27T07:05:15.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- 172.21.15.12:0/3641714773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 msgr2=0x7fb6140b7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 --2- 172.21.15.12:0/3641714773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 0x7fb6140b7f60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb604009870 tx=0x7fb60401c400 comp rx=0 tx=0).stop 2024-04-27T07:05:15.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- 172.21.15.12:0/3641714773 shutdown_connections 2024-04-27T07:05:15.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 --2- 172.21.15.12:0/3641714773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 0x7fb6140b7f60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- 172.21.15.12:0/3641714773 >> 172.21.15.12:0/3641714773 conn(0x7fb614073c00 msgr2=0x7fb6140bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:15.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- 172.21.15.12:0/3641714773 shutdown_connections 2024-04-27T07:05:15.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- 172.21.15.12:0/3641714773 wait complete. 2024-04-27T07:05:15.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 Processor -- start 2024-04-27T07:05:15.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- start start 2024-04-27T07:05:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 0x7fb61413d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb6140b64c0 0x7fb614145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb614148810 0x7fb6141e0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb6141421e0 con 0x7fb6140b5ae0 2024-04-27T07:05:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb614142360 con 0x7fb614148810 2024-04-27T07:05:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fb614156580 con 0x7fb6140b64c0 2024-04-27T07:05:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb6198a0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 0x7fb61413d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb6198a0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 0x7fb61413d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35332/0 (socket says 172.21.15.12:35332) 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb6198a0640 1 -- 172.21.15.12:0/2326624052 learned_addr learned my addr 172.21.15.12:0/2326624052 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb6198a0640 1 -- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb6140b64c0 msgr2=0x7fb614145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb6198a0640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb6140b64c0 0x7fb614145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb6198a0640 1 -- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb614148810 msgr2=0x7fb6141e0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61a0a1640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb614148810 0x7fb6141e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb6198a0640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb614148810 0x7fb6141e0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb6198a0640 1 -- 172.21.15.12:0/2326624052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb614142800 con 0x7fb6140b5ae0 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb6198a0640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 0x7fb61413d5d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb61000c480 tx=0x7fb61000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb60a7fc640 1 -- 172.21.15.12:0/2326624052 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610012830 con 0x7fb6140b5ae0 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb60a7fc640 1 -- 172.21.15.12:0/2326624052 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb610017070 con 0x7fb6140b5ae0 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- 172.21.15.12:0/2326624052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61413ce10 con 0x7fb6140b5ae0 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb60a7fc640 1 -- 172.21.15.12:0/2326624052 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610012c30 con 0x7fb6140b5ae0 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- 172.21.15.12:0/2326624052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61413deb0 con 0x7fb6140b5ae0 2024-04-27T07:05:15.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.512+0000 7fb61bb04640 1 -- 172.21.15.12:0/2326624052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb614068c20 con 0x7fb6140b5ae0 2024-04-27T07:05:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.516+0000 7fb60a7fc640 1 -- 172.21.15.12:0/2326624052 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb61001c460 con 0x7fb6140b5ae0 2024-04-27T07:05:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.516+0000 7fb60a7fc640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb5f8068380 0x7fb5f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.516+0000 7fb60a7fc640 1 -- 172.21.15.12:0/2326624052 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fb610086890 con 0x7fb6140b5ae0 2024-04-27T07:05:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.516+0000 7fb61909f640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb5f8068380 0x7fb5f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.516+0000 7fb61909f640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb5f8068380 0x7fb5f806a840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb6040ae000 tx=0x7fb604008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:15.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.520+0000 7fb60a7fc640 1 -- 172.21.15.12:0/2326624052 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb610056e40 con 0x7fb6140b5ae0 2024-04-27T07:05:15.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.680+0000 7fb61bb04640 1 -- 172.21.15.12:0/2326624052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6140bb6a0 con 0x7fb6140b5ae0 2024-04-27T07:05:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.684+0000 7fb60a7fc640 1 -- 172.21.15.12:0/2326624052 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16609 (secure 0 0 0) 0x7fb610056740 con 0x7fb6140b5ae0 2024-04-27T07:05:15.691 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:15.691 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:05:02.565909+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-27T07:04:59.279361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-27T07:04:59.893681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:05:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 -- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb5f8068380 msgr2=0x7fb5f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb5f8068380 0x7fb5f806a840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb6040ae000 tx=0x7fb604008040 comp rx=0 tx=0).stop 2024-04-27T07:05:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 -- 172.21.15.12:0/2326624052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 msgr2=0x7fb61413d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 0x7fb61413d5d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb61000c480 tx=0x7fb61000c940 comp rx=0 tx=0).stop 2024-04-27T07:05:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 -- 172.21.15.12:0/2326624052 shutdown_connections 2024-04-27T07:05:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fb5f8068380 0x7fb5f806a840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb614148810 0x7fb6141e0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fb6140b64c0 0x7fb614145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 --2- 172.21.15.12:0/2326624052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6140b5ae0 0x7fb61413d5d0 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb61000c480 tx=0x7fb61000c940 comp rx=0 tx=0).stop 2024-04-27T07:05:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 -- 172.21.15.12:0/2326624052 >> 172.21.15.12:0/2326624052 conn(0x7fb614073c00 msgr2=0x7fb614140b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 -- 172.21.15.12:0/2326624052 shutdown_connections 2024-04-27T07:05:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.688+0000 7fb61bb04640 1 -- 172.21.15.12:0/2326624052 wait complete. 2024-04-27T07:05:15.711 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-27T07:05:15.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fda09b36640 1 Processor -- start 2024-04-27T07:05:15.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fda09b36640 1 -- start start 2024-04-27T07:05:15.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fda09b36640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fda0406e370 0x7fda0413faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fda09b36640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda040b3650 0x7fda040b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fda09b36640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda040b4d70 con 0x7fda0406d300 2024-04-27T07:05:15.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fda09b36640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fda040740d0 con 0x7fda040b3650 2024-04-27T07:05:15.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fda09b36640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fda040b2bf0 con 0x7fda0406e370 2024-04-27T07:05:15.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fd9faffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda040b3650 0x7fda040b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fd9faffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fda040b3650 0x7fda040b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35336/0 (socket says 172.21.15.12:35336) 2024-04-27T07:05:15.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fd9faffd640 1 -- 172.21.15.12:0/648845404 learned_addr learned my addr 172.21.15.12:0/648845404 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:15.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fda037fe640 1 --2- 172.21.15.12:0/648845404 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fda0406e370 0x7fda0413faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fd9faffd640 1 -- 172.21.15.12:0/648845404 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fda0406e370 msgr2=0x7fda0413faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:15.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fd9faffd640 1 --2- 172.21.15.12:0/648845404 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fda0406e370 0x7fda0413faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fd9faffd640 1 -- 172.21.15.12:0/648845404 >> v1:172.21.15.53:6789/0 conn(0x7fda0406d300 legacy=0x7fda040b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:15.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.824+0000 7fd9faffd640 1 -- 172.21.15.12:0/648845404 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda04073e60 con 0x7fda040b3650 2024-04-27T07:05:15.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fd9faffd640 1 --2- 172.21.15.12:0/648845404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 0x7fda040b5190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009870 tx=0x7fd9ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898e1f6a7d2aeced server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:15.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda02ffd640 1 -- 172.21.15.12:0/648845404 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0b0070 con 0x7fda040b3650 2024-04-27T07:05:15.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda02ffd640 1 -- 172.21.15.12:0/648845404 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ec0ab600 con 0x7fda040b3650 2024-04-27T07:05:15.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda02ffd640 1 -- 172.21.15.12:0/648845404 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0ab9d0 con 0x7fda040b3650 2024-04-27T07:05:15.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 -- 172.21.15.12:0/648845404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 msgr2=0x7fda040b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 --2- 172.21.15.12:0/648845404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 0x7fda040b5190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009870 tx=0x7fd9ec01c490 comp rx=0 tx=0).stop 2024-04-27T07:05:15.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 -- 172.21.15.12:0/648845404 shutdown_connections 2024-04-27T07:05:15.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 --2- 172.21.15.12:0/648845404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 0x7fda040b5190 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 --2- 172.21.15.12:0/648845404 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fda0406e370 0x7fda0413faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 -- 172.21.15.12:0/648845404 >> 172.21.15.12:0/648845404 conn(0x7fda040b93b0 msgr2=0x7fda040b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:15.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 -- 172.21.15.12:0/648845404 shutdown_connections 2024-04-27T07:05:15.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 -- 172.21.15.12:0/648845404 wait complete. 2024-04-27T07:05:15.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 Processor -- start 2024-04-27T07:05:15.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 -- start start 2024-04-27T07:05:15.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0406d300 0x7fda0409c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fda0406e370 0x7fda041e1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 0x7fda041e2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fda041535c0 con 0x7fda040b3650 2024-04-27T07:05:15.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fda04153440 con 0x7fda0406d300 2024-04-27T07:05:15.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda09b36640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fda04153740 con 0x7fda0406e370 2024-04-27T07:05:15.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda03fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 0x7fda041e2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda03fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 0x7fda041e2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35344/0 (socket says 172.21.15.12:35344) 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda03fff640 1 -- 172.21.15.12:0/2122000060 learned_addr learned my addr 172.21.15.12:0/2122000060 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda037fe640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0406d300 0x7fda0409c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fd9faffd640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fda0406e370 0x7fda041e1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda03fff640 1 -- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fda0406e370 msgr2=0x7fda041e1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda03fff640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fda0406e370 0x7fda041e1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda03fff640 1 -- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0406d300 msgr2=0x7fda0409c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda03fff640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0406d300 0x7fda0409c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda03fff640 1 -- 172.21.15.12:0/2122000060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda041ded70 con 0x7fda040b3650 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.828+0000 7fda03fff640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 0x7fda041e2200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00be60 tx=0x7fd9fc00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda00ff9640 1 -- 172.21.15.12:0/2122000060 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc019070 con 0x7fda040b3650 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda00ff9640 1 -- 172.21.15.12:0/2122000060 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9fc014440 con 0x7fda040b3650 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda09b36640 1 -- 172.21.15.12:0/2122000060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0409dcb0 con 0x7fda040b3650 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda00ff9640 1 -- 172.21.15.12:0/2122000060 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc014830 con 0x7fda040b3650 2024-04-27T07:05:15.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda09b36640 1 -- 172.21.15.12:0/2122000060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0409d9d0 con 0x7fda040b3650 2024-04-27T07:05:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda09b36640 1 -- 172.21.15.12:0/2122000060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda04068c20 con 0x7fda040b3650 2024-04-27T07:05:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda00ff9640 1 -- 172.21.15.12:0/2122000060 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd9fc01c070 con 0x7fda040b3650 2024-04-27T07:05:15.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda00ff9640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd9e4068380 0x7fd9e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda00ff9640 1 -- 172.21.15.12:0/2122000060 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fd9fc086900 con 0x7fda040b3650 2024-04-27T07:05:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda037fe640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd9e4068380 0x7fd9e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:15.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.832+0000 7fda037fe640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd9e4068380 0x7fd9e406a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400b9d0 tx=0x7fd9f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:15.836+0000 7fda00ff9640 1 -- 172.21.15.12:0/2122000060 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd9fc056ec0 con 0x7fda040b3650 2024-04-27T07:05:16.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.032+0000 7fda09b36640 1 -- 172.21.15.12:0/2122000060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fda040b7350 con 0x7fda040b3650 2024-04-27T07:05:16.040 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-04-27T07:05:16.032+0000 7efe6513c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T07:05:16.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.036+0000 7fda00ff9640 1 -- 172.21.15.12:0/2122000060 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7fd9fc0567c0 con 0x7fda040b3650 2024-04-27T07:05:16.042 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-27T07:05:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 -- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd9e4068380 msgr2=0x7fd9e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd9e4068380 0x7fd9e406a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400b9d0 tx=0x7fd9f4008040 comp rx=0 tx=0).stop 2024-04-27T07:05:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 -- 172.21.15.12:0/2122000060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 msgr2=0x7fda041e2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 0x7fda041e2200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00be60 tx=0x7fd9fc00bf50 comp rx=0 tx=0).stop 2024-04-27T07:05:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 -- 172.21.15.12:0/2122000060 shutdown_connections 2024-04-27T07:05:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fd9e4068380 0x7fd9e406a840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:16.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda040b3650 0x7fda041e2200 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fda0406e370 0x7fda041e1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 --2- 172.21.15.12:0/2122000060 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda0406d300 0x7fda0409c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 -- 172.21.15.12:0/2122000060 >> 172.21.15.12:0/2122000060 conn(0x7fda040b93b0 msgr2=0x7fda040ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 -- 172.21.15.12:0/2122000060 shutdown_connections 2024-04-27T07:05:16.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.040+0000 7fda09b36640 1 -- 172.21.15.12:0/2122000060 wait complete. 2024-04-27T07:05:16.048 INFO:tasks.ceph.mds.a.smithi012.stderr: -2412> 2024-04-27T07:05:06.380+0000 7f29a960b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-27T07:05:16.064 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-27T07:05:16.086 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2024-04-27T07:05:16.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8493ccb640 1 Processor -- start 2024-04-27T07:05:16.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8493ccb640 1 -- start start 2024-04-27T07:05:16.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8493ccb640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f848c0b74b0 0x7f848c06f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:16.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8493ccb640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c06ea70 0x7f848c0b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:16.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8493ccb640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f848c0ba4f0 con 0x7f848c145e10 2024-04-27T07:05:16.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8493ccb640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f848c06fd30 con 0x7f848c0b74b0 2024-04-27T07:05:16.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8493ccb640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f848c06eef0 con 0x7f848c06ea70 2024-04-27T07:05:16.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8491a67640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f848c0b74b0 0x7f848c06f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:16.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8491266640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c06ea70 0x7f848c0b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:16.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8491a67640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f848c0b74b0 0x7f848c06f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.12:54348/0 (socket says 172.21.15.12:54348) 2024-04-27T07:05:16.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8491266640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c06ea70 0x7f848c0b9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3301/0 says I am v2:172.21.15.12:37656/0 (socket says 172.21.15.12:37656) 2024-04-27T07:05:16.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.180+0000 7f8491a67640 1 -- 172.21.15.12:0/1211190000 learned_addr learned my addr 172.21.15.12:0/1211190000 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:16.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8491a67640 1 -- 172.21.15.12:0/1211190000 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c06ea70 msgr2=0x7f848c0b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:16.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8491a67640 1 --2- 172.21.15.12:0/1211190000 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c06ea70 0x7f848c0b9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:16.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8491a67640 1 -- 172.21.15.12:0/1211190000 >> v1:172.21.15.53:6789/0 conn(0x7f848c145e10 legacy=0x7f848c1462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:16.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8491a67640 1 -- 172.21.15.12:0/1211190000 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c06f9c0 con 0x7f848c0b74b0 2024-04-27T07:05:16.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8491266640 1 --2- 172.21.15.12:0/1211190000 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c06ea70 0x7f848c0b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T07:05:16.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8491a67640 1 --2- 172.21.15.12:0/1211190000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0b74b0 0x7f848c06f140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8480009870 tx=0x7f848001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41e2093936101fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:16.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8490a65640 1 -- 172.21.15.12:0/1211190000 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800b2070 con 0x7f848c0b74b0 2024-04-27T07:05:16.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8490a65640 1 -- 172.21.15.12:0/1211190000 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84800a9070 con 0x7f848c0b74b0 2024-04-27T07:05:16.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8490a65640 1 -- 172.21.15.12:0/1211190000 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800ad880 con 0x7f848c0b74b0 2024-04-27T07:05:16.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8493ccb640 1 -- 172.21.15.12:0/1211190000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0b74b0 msgr2=0x7f848c06f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:16.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8493ccb640 1 --2- 172.21.15.12:0/1211190000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0b74b0 0x7f848c06f140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8480009870 tx=0x7f848001c560 comp rx=0 tx=0).stop 2024-04-27T07:05:16.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8493ccb640 1 -- 172.21.15.12:0/1211190000 shutdown_connections 2024-04-27T07:05:16.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8493ccb640 1 --2- 172.21.15.12:0/1211190000 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c06ea70 0x7f848c0b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:16.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8493ccb640 1 --2- 172.21.15.12:0/1211190000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c0b74b0 0x7f848c06f140 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:16.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8493ccb640 1 -- 172.21.15.12:0/1211190000 >> 172.21.15.12:0/1211190000 conn(0x7f848c0ba6d0 msgr2=0x7f848c0baab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:16.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8493ccb640 1 -- 172.21.15.12:0/1211190000 shutdown_connections 2024-04-27T07:05:16.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.184+0000 7f8493ccb640 1 -- 172.21.15.12:0/1211190000 wait complete. 2024-04-27T07:05:16.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 Processor -- start 2024-04-27T07:05:16.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 -- start start 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c06ea70 0x7f848c1e3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c0b74b0 0x7f848c09f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c145e10 0x7f848c142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f848c152f00 con 0x7f848c145e10 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f848c0ba0b0 con 0x7f848c06ea70 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f848c153080 con 0x7f848c0b74b0 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8492268640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c145e10 0x7f848c142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8492268640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c145e10 0x7f848c142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35372/0 (socket says 172.21.15.12:35372) 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8492268640 1 -- 172.21.15.12:0/4080573360 learned_addr learned my addr 172.21.15.12:0/4080573360 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8491266640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c0b74b0 0x7f848c09f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8491a67640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c06ea70 0x7f848c1e3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8492268640 1 -- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c0b74b0 msgr2=0x7f848c09f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8492268640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c0b74b0 0x7f848c09f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8492268640 1 -- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c06ea70 msgr2=0x7f848c1e3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8492268640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c06ea70 0x7f848c1e3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8492268640 1 -- 172.21.15.12:0/4080573360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c143550 con 0x7f848c145e10 2024-04-27T07:05:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8491a67640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c06ea70 0x7f848c1e3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8492268640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c145e10 0x7f848c142370 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f848400c730 tx=0x7f848400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f847e7fc640 1 -- 172.21.15.12:0/4080573360 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484003b80 con 0x7f848c145e10 2024-04-27T07:05:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f847e7fc640 1 -- 172.21.15.12:0/4080573360 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8484003d20 con 0x7f848c145e10 2024-04-27T07:05:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 -- 172.21.15.12:0/4080573360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848c0b1590 con 0x7f848c145e10 2024-04-27T07:05:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f847e7fc640 1 -- 172.21.15.12:0/4080573360 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848400aa30 con 0x7f848c145e10 2024-04-27T07:05:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 -- 172.21.15.12:0/4080573360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848c0b7070 con 0x7f848c145e10 2024-04-27T07:05:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f8493ccb640 1 -- 172.21.15.12:0/4080573360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f848c069330 con 0x7f848c145e10 2024-04-27T07:05:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.188+0000 7f847e7fc640 1 -- 172.21.15.12:0/4080573360 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f848400abd0 con 0x7f848c145e10 2024-04-27T07:05:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.192+0000 7f847e7fc640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8470068380 0x7f847006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.192+0000 7f847e7fc640 1 -- 172.21.15.12:0/4080573360 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5795+0+0 (secure 0 0 0) 0x7f84840868b0 con 0x7f848c145e10 2024-04-27T07:05:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.192+0000 7f8491a67640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8470068380 0x7f847006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.192+0000 7f8491a67640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8470068380 0x7f847006a840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f84800af000 tx=0x7f84800a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:16.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.196+0000 7f847e7fc640 1 -- 172.21.15.12:0/4080573360 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f848c0ba4f0 con 0x7f848c145e10 2024-04-27T07:05:16.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:16.392+0000 7f8493ccb640 1 -- 172.21.15.12:0/4080573360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f848c144ff0 con 0x7f848c145e10 2024-04-27T07:05:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.040+0000 7f847e7fc640 1 -- 172.21.15.12:0/4080573360 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f8484056e20 con 0x7f848c145e10 2024-04-27T07:05:17.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.040+0000 7f8493ccb640 1 -- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8470068380 msgr2=0x7f847006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.040+0000 7f8493ccb640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8470068380 0x7f847006a840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f84800af000 tx=0x7f84800a7000 comp rx=0 tx=0).stop 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.040+0000 7f8493ccb640 1 -- 172.21.15.12:0/4080573360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c145e10 msgr2=0x7f848c142370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.040+0000 7f8493ccb640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c145e10 0x7f848c142370 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f848400c730 tx=0x7f848400cbf0 comp rx=0 tx=0).stop 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.044+0000 7f8493ccb640 1 -- 172.21.15.12:0/4080573360 shutdown_connections 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.044+0000 7f8493ccb640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8470068380 0x7f847006a840 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.044+0000 7f8493ccb640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f848c145e10 0x7f848c142370 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.044+0000 7f8493ccb640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f848c0b74b0 0x7f848c09f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.044+0000 7f8493ccb640 1 --2- 172.21.15.12:0/4080573360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f848c06ea70 0x7f848c1e3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.044+0000 7f8493ccb640 1 -- 172.21.15.12:0/4080573360 >> 172.21.15.12:0/4080573360 conn(0x7f848c0ba6d0 msgr2=0x7f848c09bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.044+0000 7f8493ccb640 1 -- 172.21.15.12:0/4080573360 shutdown_connections 2024-04-27T07:05:17.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.044+0000 7f8493ccb640 1 -- 172.21.15.12:0/4080573360 wait complete. 2024-04-27T07:05:17.070 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-27T07:05:17.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f452704f640 1 Processor -- start 2024-04-27T07:05:17.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f452704f640 1 -- start start 2024-04-27T07:05:17.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f452704f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f452006d500 0x7f452009e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:17.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f452704f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f45200b4550 0x7f4520152c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:17.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f452704f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45200afcb0 con 0x7f452006d8e0 2024-04-27T07:05:17.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f452704f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f45200ae740 con 0x7f452006d500 2024-04-27T07:05:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f452704f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f45200aead0 con 0x7f45200b4550 2024-04-27T07:05:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f4524deb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f452006d500 0x7f452009e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f4524deb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f452006d500 0x7f452009e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35380/0 (socket says 172.21.15.12:35380) 2024-04-27T07:05:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f4524deb640 1 -- 172.21.15.12:0/2089542563 learned_addr learned my addr 172.21.15.12:0/2089542563 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f45255ec640 1 --2- 172.21.15.12:0/2089542563 >> v2:172.21.15.53:3300/0 conn(0x7f45200b4550 0x7f4520152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f4524deb640 1 -- 172.21.15.12:0/2089542563 >> v2:172.21.15.53:3300/0 conn(0x7f45200b4550 msgr2=0x7f4520152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f4524deb640 1 --2- 172.21.15.12:0/2089542563 >> v2:172.21.15.53:3300/0 conn(0x7f45200b4550 0x7f4520152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f4524deb640 1 -- 172.21.15.12:0/2089542563 >> v1:172.21.15.53:6789/0 conn(0x7f452006d8e0 legacy=0x7f45200b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f4524deb640 1 -- 172.21.15.12:0/2089542563 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45200740d0 con 0x7f452006d500 2024-04-27T07:05:17.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.240+0000 7f45255ec640 1 --2- 172.21.15.12:0/2089542563 >> v2:172.21.15.53:3300/0 conn(0x7f45200b4550 0x7f4520152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:17.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f4524deb640 1 --2- 172.21.15.12:0/2089542563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d500 0x7f452009e2a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f452006ee20 tx=0x7f451001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48925567dc054cf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f451f7fe640 1 -- 172.21.15.12:0/2089542563 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100b0070 con 0x7f452006d500 2024-04-27T07:05:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f451f7fe640 1 -- 172.21.15.12:0/2089542563 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45100ab600 con 0x7f452006d500 2024-04-27T07:05:17.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f451f7fe640 1 -- 172.21.15.12:0/2089542563 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100ab9d0 con 0x7f452006d500 2024-04-27T07:05:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f452704f640 1 -- 172.21.15.12:0/2089542563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d500 msgr2=0x7f452009e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f452704f640 1 --2- 172.21.15.12:0/2089542563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d500 0x7f452009e2a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f452006ee20 tx=0x7f451001c450 comp rx=0 tx=0).stop 2024-04-27T07:05:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f452704f640 1 -- 172.21.15.12:0/2089542563 shutdown_connections 2024-04-27T07:05:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f452704f640 1 --2- 172.21.15.12:0/2089542563 >> v2:172.21.15.53:3300/0 conn(0x7f45200b4550 0x7f4520152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f452704f640 1 --2- 172.21.15.12:0/2089542563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d500 0x7f452009e2a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f452704f640 1 -- 172.21.15.12:0/2089542563 >> 172.21.15.12:0/2089542563 conn(0x7f45200b9840 msgr2=0x7f45200b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f452704f640 1 -- 172.21.15.12:0/2089542563 shutdown_connections 2024-04-27T07:05:17.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.244+0000 7f452704f640 1 -- 172.21.15.12:0/2089542563 wait complete. 2024-04-27T07:05:17.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 Processor -- start 2024-04-27T07:05:17.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 -- start start 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f452006d500 0x7f45200b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d8e0 0x7f45201e4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45200b4550 0x7f45201e0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f452009e960 con 0x7f452006d8e0 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f452009e7e0 con 0x7f45200b4550 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f45200b5b90 con 0x7f452006d500 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d8e0 0x7f45201e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d8e0 0x7f45201e4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35388/0 (socket says 172.21.15.12:35388) 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451ffff640 1 -- 172.21.15.12:0/2671881915 learned_addr learned my addr 172.21.15.12:0/2671881915 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f4524deb640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f452006d500 0x7f45200b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:17.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f45255ec640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45200b4550 0x7f45201e0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451ffff640 1 -- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f452006d500 msgr2=0x7f45200b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451ffff640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f452006d500 0x7f45200b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451ffff640 1 -- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45200b4550 msgr2=0x7f45201e0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451ffff640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45200b4550 0x7f45201e0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451ffff640 1 -- 172.21.15.12:0/2671881915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4520150660 con 0x7f452006d8e0 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f4524deb640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f452006d500 0x7f45200b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f45255ec640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45200b4550 0x7f45201e0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451ffff640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d8e0 0x7f45201e4390 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f451400c410 tx=0x7f451400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451d7fa640 1 -- 172.21.15.12:0/2671881915 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514011790 con 0x7f452006d8e0 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451d7fa640 1 -- 172.21.15.12:0/2671881915 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4514017070 con 0x7f452006d8e0 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451d7fa640 1 -- 172.21.15.12:0/2671881915 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514011bb0 con 0x7f452006d8e0 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 -- 172.21.15.12:0/2671881915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452014fb60 con 0x7f452006d8e0 2024-04-27T07:05:17.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 -- 172.21.15.12:0/2671881915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4520150bb0 con 0x7f452006d8e0 2024-04-27T07:05:17.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f452704f640 1 -- 172.21.15.12:0/2671881915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4520068d00 con 0x7f452006d8e0 2024-04-27T07:05:17.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.248+0000 7f451d7fa640 1 -- 172.21.15.12:0/2671881915 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f451401c460 con 0x7f452006d8e0 2024-04-27T07:05:17.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.252+0000 7f451d7fa640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f450c068380 0x7f450c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:17.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.252+0000 7f451d7fa640 1 -- 172.21.15.12:0/2671881915 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5795+0+0 (secure 0 0 0) 0x7f4514086600 con 0x7f452006d8e0 2024-04-27T07:05:17.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.252+0000 7f4524deb640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f450c068380 0x7f450c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:17.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.252+0000 7f4524deb640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f450c068380 0x7f450c06a840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f452006ed30 tx=0x7f4510009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:17.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.256+0000 7f451d7fa640 1 -- 172.21.15.12:0/2671881915 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4514021020 con 0x7f452006d8e0 2024-04-27T07:05:17.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:17.416+0000 7f452704f640 1 -- 172.21.15.12:0/2671881915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f452006e960 con 0x7f452006d8e0 2024-04-27T07:05:18.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.048+0000 7f451d7fa640 1 -- 172.21.15.12:0/2671881915 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f4514056bf0 con 0x7f452006d8e0 2024-04-27T07:05:18.054 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2024-04-27T07:05:18.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 -- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f450c068380 msgr2=0x7f450c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:18.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f450c068380 0x7f450c06a840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f452006ed30 tx=0x7f4510009040 comp rx=0 tx=0).stop 2024-04-27T07:05:18.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 -- 172.21.15.12:0/2671881915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d8e0 msgr2=0x7f45201e4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:18.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d8e0 0x7f45201e4390 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f451400c410 tx=0x7f451400c8d0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 -- 172.21.15.12:0/2671881915 shutdown_connections 2024-04-27T07:05:18.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f450c068380 0x7f450c06a840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f45200b4550 0x7f45201e0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f452006d8e0 0x7f45201e4390 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 --2- 172.21.15.12:0/2671881915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f452006d500 0x7f45200b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 -- 172.21.15.12:0/2671881915 >> 172.21.15.12:0/2671881915 conn(0x7f45200b9840 msgr2=0x7f45200b8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:18.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.052+0000 7f452704f640 1 -- 172.21.15.12:0/2671881915 shutdown_connections 2024-04-27T07:05:18.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.056+0000 7f452704f640 1 -- 172.21.15.12:0/2671881915 wait complete. 2024-04-27T07:05:18.080 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-27T07:05:18.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3efcc18640 1 Processor -- start 2024-04-27T07:05:18.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3efcc18640 1 -- start start 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3efcc18640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3ef806bde0 0x7f3ef8149370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3efcc18640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef809c7b0 0x7f3ef809cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3efcc18640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f3ef806e460 0x7f3ef806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3efcc18640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef80b5640 con 0x7f3ef806bde0 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3efcc18640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef806c1c0 con 0x7f3ef806e460 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3efcc18640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef80b5ad0 con 0x7f3ef809c7b0 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3ef806bde0 0x7f3ef8149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3ef806bde0 0x7f3ef8149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41126/0 (socket says 172.21.15.12:41126) 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef659c640 1 -- 172.21.15.12:0/1358262283 learned_addr learned my addr 172.21.15.12:0/1358262283 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef6d9d640 1 --2- 172.21.15.12:0/1358262283 >> v2:172.21.15.53:3300/0 conn(0x7f3ef806e460 0x7f3ef806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef5d9b640 1 --2- 172.21.15.12:0/1358262283 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef809c7b0 0x7f3ef809cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:18.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef659c640 1 -- 172.21.15.12:0/1358262283 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef809c7b0 msgr2=0x7f3ef809cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:18.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef659c640 1 --2- 172.21.15.12:0/1358262283 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef809c7b0 0x7f3ef809cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef659c640 1 -- 172.21.15.12:0/1358262283 >> v2:172.21.15.53:3300/0 conn(0x7f3ef806e460 msgr2=0x7f3ef806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:18.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef659c640 1 --2- 172.21.15.12:0/1358262283 >> v2:172.21.15.53:3300/0 conn(0x7f3ef806e460 0x7f3ef806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef659c640 1 -- 172.21.15.12:0/1358262283 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef80b5db0 con 0x7f3ef806bde0 2024-04-27T07:05:18.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef6d9d640 1 --2- 172.21.15.12:0/1358262283 >> v2:172.21.15.53:3300/0 conn(0x7f3ef806e460 0x7f3ef806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:18.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.204+0000 7f3ef5d9b640 1 --2- 172.21.15.12:0/1358262283 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef809c7b0 0x7f3ef809cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:18.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3ef659c640 1 --2- 172.21.15.12:0/1358262283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef806bde0 0x7f3ef8149370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3ef809fc50 tx=0x7f3ee801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b9cb2d9cadaa92 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:18.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3ef559a640 1 -- 172.21.15.12:0/1358262283 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80b1070 con 0x7f3ef806bde0 2024-04-27T07:05:18.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3ef559a640 1 -- 172.21.15.12:0/1358262283 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee80ac630 con 0x7f3ef806bde0 2024-04-27T07:05:18.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3ef559a640 1 -- 172.21.15.12:0/1358262283 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80a8050 con 0x7f3ef806bde0 2024-04-27T07:05:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 -- 172.21.15.12:0/1358262283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef806bde0 msgr2=0x7f3ef8149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 --2- 172.21.15.12:0/1358262283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef806bde0 0x7f3ef8149370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3ef809fc50 tx=0x7f3ee801c520 comp rx=0 tx=0).stop 2024-04-27T07:05:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 -- 172.21.15.12:0/1358262283 shutdown_connections 2024-04-27T07:05:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 --2- 172.21.15.12:0/1358262283 >> v2:172.21.15.53:3300/0 conn(0x7f3ef806e460 0x7f3ef806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 --2- 172.21.15.12:0/1358262283 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef809c7b0 0x7f3ef809cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 --2- 172.21.15.12:0/1358262283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef806bde0 0x7f3ef8149370 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 -- 172.21.15.12:0/1358262283 >> 172.21.15.12:0/1358262283 conn(0x7f3ef806d550 msgr2=0x7f3ef80ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 -- 172.21.15.12:0/1358262283 shutdown_connections 2024-04-27T07:05:18.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 -- 172.21.15.12:0/1358262283 wait complete. 2024-04-27T07:05:18.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 Processor -- start 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.208+0000 7f3efcc18640 1 -- start start 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3efcc18640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef806bde0 0x7f3ef81404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3efcc18640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ef806e460 0x7f3ef809df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3efcc18640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef809c7b0 0x7f3ef809e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3efcc18640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef806c9d0 con 0x7f3ef809c7b0 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3efcc18640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef809d130 con 0x7f3ef806e460 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3efcc18640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef80b65b0 con 0x7f3ef806bde0 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef809c7b0 0x7f3ef809e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef809c7b0 0x7f3ef809e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41134/0 (socket says 172.21.15.12:41134) 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef6d9d640 1 -- 172.21.15.12:0/3612795823 learned_addr learned my addr 172.21.15.12:0/3612795823 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef5d9b640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ef806e460 0x7f3ef809df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef659c640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef806bde0 0x7f3ef81404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef6d9d640 1 -- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef806bde0 msgr2=0x7f3ef81404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:18.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef6d9d640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef806bde0 0x7f3ef81404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef6d9d640 1 -- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ef806e460 msgr2=0x7f3ef809df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef6d9d640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ef806e460 0x7f3ef809df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef6d9d640 1 -- 172.21.15.12:0/3612795823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef81500c0 con 0x7f3ef809c7b0 2024-04-27T07:05:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef6d9d640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef809c7b0 0x7f3ef809e4a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000cf20 tx=0x7f3ef000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ee6ffd640 1 -- 172.21.15.12:0/3612795823 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef001a070 con 0x7f3ef809c7b0 2024-04-27T07:05:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ee6ffd640 1 -- 172.21.15.12:0/3612795823 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef000d840 con 0x7f3ef809c7b0 2024-04-27T07:05:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ee6ffd640 1 -- 172.21.15.12:0/3612795823 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef000dc30 con 0x7f3ef809c7b0 2024-04-27T07:05:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3efcc18640 1 -- 172.21.15.12:0/3612795823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef809f360 con 0x7f3ef809c7b0 2024-04-27T07:05:18.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3efcc18640 1 -- 172.21.15.12:0/3612795823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef81503d0 con 0x7f3ef809c7b0 2024-04-27T07:05:18.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3efcc18640 1 -- 172.21.15.12:0/3612795823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef814fec0 con 0x7f3ef809c7b0 2024-04-27T07:05:18.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ee6ffd640 1 -- 172.21.15.12:0/3612795823 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3ef001d070 con 0x7f3ef809c7b0 2024-04-27T07:05:18.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ee6ffd640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ed8068380 0x7f3ed806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:18.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ee6ffd640 1 -- 172.21.15.12:0/3612795823 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5357+0+0 (secure 0 0 0) 0x7f3ef0087000 con 0x7f3ef809c7b0 2024-04-27T07:05:18.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.212+0000 7f3ef659c640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ed8068380 0x7f3ed806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:18.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.216+0000 7f3ef659c640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ed8068380 0x7f3ed806a840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80ae000 tx=0x7f3ee8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:18.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.216+0000 7f3ee6ffd640 1 -- 172.21.15.12:0/3612795823 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3ef00577b0 con 0x7f3ef809c7b0 2024-04-27T07:05:18.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:18.380+0000 7f3efcc18640 1 -- 172.21.15.12:0/3612795823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3ef8069330 con 0x7f3ef809c7b0 2024-04-27T07:05:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.056+0000 7f3ee6ffd640 1 -- 172.21.15.12:0/3612795823 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7f3ef00570b0 con 0x7f3ef809c7b0 2024-04-27T07:05:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2024-04-27T07:05:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 -- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ed8068380 msgr2=0x7f3ed806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ed8068380 0x7f3ed806a840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80ae000 tx=0x7f3ee8008040 comp rx=0 tx=0).stop 2024-04-27T07:05:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 -- 172.21.15.12:0/3612795823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef809c7b0 msgr2=0x7f3ef809e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef809c7b0 0x7f3ef809e4a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000cf20 tx=0x7f3ef000a710 comp rx=0 tx=0).stop 2024-04-27T07:05:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 -- 172.21.15.12:0/3612795823 shutdown_connections 2024-04-27T07:05:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ed8068380 0x7f3ed806a840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ef809c7b0 0x7f3ef809e4a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ef806e460 0x7f3ef809df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 --2- 172.21.15.12:0/3612795823 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3ef806bde0 0x7f3ef81404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 -- 172.21.15.12:0/3612795823 >> 172.21.15.12:0/3612795823 conn(0x7f3ef806d550 msgr2=0x7f3ef809bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:19.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 -- 172.21.15.12:0/3612795823 shutdown_connections 2024-04-27T07:05:19.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.060+0000 7f3efcc18640 1 -- 172.21.15.12:0/3612795823 wait complete. 2024-04-27T07:05:19.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-27T07:05:19.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f384b5ff640 1 Processor -- start 2024-04-27T07:05:19.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f384b5ff640 1 -- start start 2024-04-27T07:05:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f384b5ff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38440b3650 0x7f38440b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f384b5ff640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38440b4d70 con 0x7f384406d300 2024-04-27T07:05:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f384b5ff640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3844073e60 con 0x7f384406e370 2024-04-27T07:05:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f384b5ff640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f38440740d0 con 0x7f38440b3650 2024-04-27T07:05:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f3849b9c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f384406d300 0x7f3844150420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57972/0 (socket says 172.21.15.12:57972) 2024-04-27T07:05:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f3849b9c640 1 -- 172.21.15.12:0/3083689308 learned_addr learned my addr 172.21.15.12:0/3083689308 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f3848b9a640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38440b3650 0x7f38440b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 560596277 0 0) 0x7f38440b4d70 con 0x7f384406d300 2024-04-27T07:05:19.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.212+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3830003530 con 0x7f384406d300 2024-04-27T07:05:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 747535364 0 0) 0x7f3830003530 con 0x7f384406d300 2024-04-27T07:05:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38440b3650 msgr2=0x7f38440b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38440b3650 0x7f38440b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 >> v1:172.21.15.53:6789/0 conn(0x7f384406e370 legacy=0x7f384413faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38440b2bf0 con 0x7f384406d300 2024-04-27T07:05:19.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f383c001a00 con 0x7f384406d300 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 >> v1:172.21.15.12:6789/0 conn(0x7f384406d300 legacy=0x7f3844150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38300039f0 0x7f3830003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38440b3650 0x7f3830004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3830004d50 0x7f38300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f38440740d0 con 0x7f3830004d50 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f383000bb00 con 0x7f38300039f0 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f383000bd00 con 0x7f38440b3650 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3849b9c640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3830004d50 0x7f38300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f384939b640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38300039f0 0x7f3830003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3848b9a640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38440b3650 0x7f3830004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3849b9c640 1 -- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38440b3650 msgr2=0x7f3830004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3849b9c640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38440b3650 0x7f3830004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3849b9c640 1 -- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38300039f0 msgr2=0x7f3830003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3849b9c640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38300039f0 0x7f3830003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3849b9c640 1 -- 172.21.15.12:0/3083689308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3830003530 con 0x7f3830004d50 2024-04-27T07:05:19.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3848b9a640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f38440b3650 0x7f3830004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3849b9c640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3830004d50 0x7f38300051d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f384406d730 tx=0x7f383c01fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14ba93ff77e7625 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c00a480 con 0x7f3830004d50 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c0aa6b0 con 0x7f3830004d50 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3837fff640 1 -- 172.21.15.12:0/3083689308 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0aaae0 con 0x7f3830004d50 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f384b5ff640 1 -- 172.21.15.12:0/3083689308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3830004d50 msgr2=0x7f38300051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f384b5ff640 1 --2- 172.21.15.12:0/3083689308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3830004d50 0x7f38300051d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f384406d730 tx=0x7f383c01fa80 comp rx=0 tx=0).stop 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f3849b9c640 1 -- 172.21.15.12:0/3083689308 reap_dead start 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f384b5ff640 1 -- 172.21.15.12:0/3083689308 shutdown_connections 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f384b5ff640 1 -- 172.21.15.12:0/3083689308 >> 172.21.15.12:0/3083689308 conn(0x7f38440b93b0 msgr2=0x7f38440b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f384b5ff640 1 -- 172.21.15.12:0/3083689308 shutdown_connections 2024-04-27T07:05:19.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.216+0000 7f384b5ff640 1 -- 172.21.15.12:0/3083689308 wait complete. 2024-04-27T07:05:19.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 Processor -- start 2024-04-27T07:05:19.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 -- start start 2024-04-27T07:05:19.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3830004d50 0x7f384409cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844139820 0x7f38441decc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3844139ca0 0x7f38441df200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f38441401b0 con 0x7f3844139ca0 2024-04-27T07:05:19.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f384406e750 con 0x7f3844139820 2024-04-27T07:05:19.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f38440b5190 con 0x7f3830004d50 2024-04-27T07:05:19.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3849b9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3844139ca0 0x7f38441df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3849b9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3844139ca0 0x7f38441df200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41158/0 (socket says 172.21.15.12:41158) 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3849b9c640 1 -- 172.21.15.12:0/1785593575 learned_addr learned my addr 172.21.15.12:0/1785593575 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3848b9a640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844139820 0x7f38441decc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384939b640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3830004d50 0x7f384409cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3849b9c640 1 -- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3830004d50 msgr2=0x7f384409cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3849b9c640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3830004d50 0x7f384409cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3849b9c640 1 -- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844139820 msgr2=0x7f38441decc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3849b9c640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844139820 0x7f38441decc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3849b9c640 1 -- 172.21.15.12:0/1785593575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384409c3f0 con 0x7f3844139ca0 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384939b640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3830004d50 0x7f384409cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3849b9c640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3844139ca0 0x7f38441df200 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f383c021160 tx=0x7f3830005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3835ffb640 1 -- 172.21.15.12:0/1785593575 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c00a480 con 0x7f3844139ca0 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3835ffb640 1 -- 172.21.15.12:0/1785593575 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c0aa4b0 con 0x7f3844139ca0 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 -- 172.21.15.12:0/1785593575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384409e2a0 con 0x7f3844139ca0 2024-04-27T07:05:19.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f3835ffb640 1 -- 172.21.15.12:0/1785593575 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0019c0 con 0x7f3844139ca0 2024-04-27T07:05:19.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 -- 172.21.15.12:0/1785593575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38441df740 con 0x7f3844139ca0 2024-04-27T07:05:19.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.220+0000 7f384b5ff640 1 -- 172.21.15.12:0/1785593575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38440b4a70 con 0x7f3844139ca0 2024-04-27T07:05:19.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.224+0000 7f3835ffb640 1 -- 172.21.15.12:0/1785593575 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f383c0bb030 con 0x7f3844139ca0 2024-04-27T07:05:19.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.224+0000 7f3835ffb640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38280683b0 0x7f382806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.224+0000 7f3835ffb640 1 -- 172.21.15.12:0/1785593575 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f383c1231c0 con 0x7f3844139ca0 2024-04-27T07:05:19.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.224+0000 7f384939b640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38280683b0 0x7f382806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.224+0000 7f384939b640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38280683b0 0x7f382806a870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3838000c10 tx=0x7f3838009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:19.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.228+0000 7f3835ffb640 1 -- 172.21.15.12:0/1785593575 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f383c0ae030 con 0x7f3844139ca0 2024-04-27T07:05:19.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.396+0000 7f384b5ff640 1 -- 172.21.15.12:0/1785593575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3844070540 con 0x7f3844139ca0 2024-04-27T07:05:19.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.400+0000 7f3835ffb640 1 -- 172.21.15.12:0/1785593575 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f383c019790 con 0x7f3844139ca0 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 -- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38280683b0 msgr2=0x7f382806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38280683b0 0x7f382806a870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3838000c10 tx=0x7f3838009040 comp rx=0 tx=0).stop 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 -- 172.21.15.12:0/1785593575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3844139ca0 msgr2=0x7f38441df200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3844139ca0 0x7f38441df200 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f383c021160 tx=0x7f3830005840 comp rx=0 tx=0).stop 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 -- 172.21.15.12:0/1785593575 shutdown_connections 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f38280683b0 0x7f382806a870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3844139ca0 0x7f38441df200 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3844139820 0x7f38441decc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 --2- 172.21.15.12:0/1785593575 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3830004d50 0x7f384409cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 -- 172.21.15.12:0/1785593575 >> 172.21.15.12:0/1785593575 conn(0x7f38440b93b0 msgr2=0x7f3844074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 -- 172.21.15.12:0/1785593575 shutdown_connections 2024-04-27T07:05:19.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.404+0000 7f384b5ff640 1 -- 172.21.15.12:0/1785593575 wait complete. 2024-04-27T07:05:19.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-04-27T07:05:19.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.544+0000 7fdc22616640 1 Processor -- start 2024-04-27T07:05:19.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.544+0000 7fdc22616640 1 -- start start 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc22616640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c067b70 0x7fdc1c148030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc22616640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc1c067f50 con 0x7fdc1c0734c0 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc22616640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc1c072b20 con 0x7fdc1c1446a0 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc22616640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1c0b1350 con 0x7fdc1c067b70 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1b7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdc1c0734c0 0x7fdc1c143f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57980/0 (socket says 172.21.15.12:57980) 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1b7fe640 1 -- 172.21.15.12:0/2558598570 learned_addr learned my addr 172.21.15.12:0/2558598570 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c067b70 0x7fdc1c148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1725500985 0 0) 0x7fdc1c067f50 con 0x7fdc1c0734c0 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbfc003530 con 0x7fdc1c0734c0 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 471475528 0 0) 0x7fdbfc003530 con 0x7fdc1c0734c0 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c067b70 msgr2=0x7fdc1c148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c067b70 0x7fdc1c148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 >> v1:172.21.15.53:6789/0 conn(0x7fdc1c1446a0 legacy=0x7fdc1c14f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c071cc0 con 0x7fdc1c0734c0 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7fdc04001680 con 0x7fdc1c0734c0 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 >> v1:172.21.15.12:6789/0 conn(0x7fdc1c0734c0 legacy=0x7fdc1c143f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c067b70 0x7fdbfc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbfc003f70 0x7fdbfc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbfc004d50 0x7fdbfc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c0b1350 con 0x7fdbfc004d50 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc00bb60 con 0x7fdbfc003f70 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fdbfc00bd60 con 0x7fdc1c067b70 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc20bb3640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbfc004d50 0x7fdbfc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c067b70 0x7fdbfc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc20bb3640 1 -- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c067b70 msgr2=0x7fdbfc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc20bb3640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c067b70 0x7fdbfc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc1b7fe640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbfc003f70 0x7fdbfc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc20bb3640 1 -- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbfc003f70 msgr2=0x7fdbfc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:19.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc20bb3640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbfc003f70 0x7fdbfc004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.548+0000 7fdc20bb3640 1 -- 172.21.15.12:0/2558598570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbfc003530 con 0x7fdbfc004d50 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbfc004d50 0x7fdbfc0051d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc14001e70 tx=0x7fdc1401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fddd8b3fc068b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14001a90 con 0x7fdbfc004d50 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc14003b80 con 0x7fdbfc004d50 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc1affd640 1 -- 172.21.15.12:0/2558598570 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14005080 con 0x7fdbfc004d50 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- 172.21.15.12:0/2558598570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbfc004d50 msgr2=0x7fdbfc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 --2- 172.21.15.12:0/2558598570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbfc004d50 0x7fdbfc0051d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdc14001e70 tx=0x7fdc1401c780 comp rx=0 tx=0).stop 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 -- 172.21.15.12:0/2558598570 reap_dead start 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- 172.21.15.12:0/2558598570 shutdown_connections 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- 172.21.15.12:0/2558598570 >> 172.21.15.12:0/2558598570 conn(0x7fdc1c06bb80 msgr2=0x7fdc1c06bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- 172.21.15.12:0/2558598570 shutdown_connections 2024-04-27T07:05:19.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- 172.21.15.12:0/2558598570 wait complete. 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 Processor -- start 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- start start 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c0b2ff0 0x7fdc1c1e0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c1d1860 0x7fdc1c0a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c13ea60 0x7fdc1c13eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c0b0b90 con 0x7fdc1c13ea60 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c072600 con 0x7fdc1c0b2ff0 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1c0b0d10 con 0x7fdc1c1d1860 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c13ea60 0x7fdc1c13eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c13ea60 0x7fdc1c13eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41178/0 (socket says 172.21.15.12:41178) 2024-04-27T07:05:19.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 -- 172.21.15.12:0/733130801 learned_addr learned my addr 172.21.15.12:0/733130801 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc1b7fe640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c1d1860 0x7fdc1c0a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c0b2ff0 0x7fdc1c1e0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 -- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c1d1860 msgr2=0x7fdc1c0a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c1d1860 0x7fdc1c0a99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 -- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c0b2ff0 msgr2=0x7fdc1c1e0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c0b2ff0 0x7fdc1c1e0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 -- 172.21.15.12:0/733130801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c13f420 con 0x7fdc1c13ea60 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c0b2ff0 0x7fdc1c1e0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc20bb3640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c13ea60 0x7fdc1c13eee0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdc14098190 tx=0x7fdbfc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc18ff9640 1 -- 172.21.15.12:0/733130801 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc140b8070 con 0x7fdc1c13ea60 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc18ff9640 1 -- 172.21.15.12:0/733130801 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc14005580 con 0x7fdc1c13ea60 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- 172.21.15.12:0/733130801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c1d2980 con 0x7fdc1c13ea60 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc18ff9640 1 -- 172.21.15.12:0/733130801 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14001a90 con 0x7fdc1c13ea60 2024-04-27T07:05:19.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.552+0000 7fdc22616640 1 -- 172.21.15.12:0/733130801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c0a9f30 con 0x7fdc1c13ea60 2024-04-27T07:05:19.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.556+0000 7fdc22616640 1 -- 172.21.15.12:0/733130801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1c069330 con 0x7fdc1c13ea60 2024-04-27T07:05:19.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.556+0000 7fdc18ff9640 1 -- 172.21.15.12:0/733130801 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdc140bb070 con 0x7fdc1c13ea60 2024-04-27T07:05:19.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.556+0000 7fdc18ff9640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdc00068380 0x7fdc0006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.556+0000 7fdc18ff9640 1 -- 172.21.15.12:0/733130801 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fdc1411f550 con 0x7fdc1c13ea60 2024-04-27T07:05:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.556+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdc00068380 0x7fdc0006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.556+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdc00068380 0x7fdc0006a840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009a20 tx=0x7fdc0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:19.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.560+0000 7fdc18ff9640 1 -- 172.21.15.12:0/733130801 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdc140ac020 con 0x7fdc1c13ea60 2024-04-27T07:05:19.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:19.780+0000 7fdc22616640 1 -- 172.21.15.12:0/733130801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7fdc1c0aa200 con 0x7fdc1c13ea60 2024-04-27T07:05:20.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.060+0000 7fdc18ff9640 1 -- 172.21.15.12:0/733130801 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v78) ==== 134+0+0 (secure 0 0 0) 0x7fdc14016490 con 0x7fdc1c13ea60 2024-04-27T07:05:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 -- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdc00068380 msgr2=0x7fdc0006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdc00068380 0x7fdc0006a840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009a20 tx=0x7fdc0c008040 comp rx=0 tx=0).stop 2024-04-27T07:05:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 -- 172.21.15.12:0/733130801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c13ea60 msgr2=0x7fdc1c13eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c13ea60 0x7fdc1c13eee0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdc14098190 tx=0x7fdbfc005840 comp rx=0 tx=0).stop 2024-04-27T07:05:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 -- 172.21.15.12:0/733130801 shutdown_connections 2024-04-27T07:05:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fdc00068380 0x7fdc0006a840 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c13ea60 0x7fdc1c13eee0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:20.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fdc1c1d1860 0x7fdc1c0a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 --2- 172.21.15.12:0/733130801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc1c0b2ff0 0x7fdc1c1e0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 -- 172.21.15.12:0/733130801 >> 172.21.15.12:0/733130801 conn(0x7fdc1c06bb80 msgr2=0x7fdc1c0b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 -- 172.21.15.12:0/733130801 shutdown_connections 2024-04-27T07:05:20.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:20.064+0000 7fdc22616640 1 -- 172.21.15.12:0/733130801 wait complete. 2024-04-27T07:05:20.092 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed ... ERROR 2024-04-27T07:05:20.092 INFO:tasks.cephfs_test_runner: 2024-04-27T07:05:20.092 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-27T07:05:20.092 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-27T07:05:20.092 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-27T07:05:20.093 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi012 with status 127: 'cephfs-shell -b /tmp/test-cephfsxv5ouxx1' 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner: 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner:Ran 1 test in 38.180s 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner: 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner: 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-04-27T07:05:20.094 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi012 with status 127: 'cephfs-shell -b /tmp/test-cephfsxv5ouxx1' 2024-04-27T07:05:20.095 INFO:tasks.cephfs_test_runner: 2024-04-27T07:05:20.096 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-27T07:05:25.017 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=669383f5e6d6454e8b01af5d51ab1fd8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_4e213a58e787437ff508e6c890ccd5fd32dc96b5/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-27T07:05:25.019 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-27T07:05:25.030 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-27T07:05:25.042 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-27T07:05:25.042 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-27T07:05:25.042 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T07:05:25.046 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:05:25.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-04-27T07:05:25.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:25.056 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-27T07:05:25.056 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:05:25.102 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-27T07:05:25.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:25.103 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T07:05:25.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T07:05:25.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-04-27T07:05:25.343 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-27T07:05:25.352 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-27T07:05:25.352 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:05:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f8574a95640 1 Processor -- start 2024-04-27T07:05:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f8574a95640 1 -- start start 2024-04-27T07:05:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f8574a95640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f857013fef0 0x7f85700b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f8574a95640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8570145ec0 0x7f8570146310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f8574a95640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8570145a00 con 0x7f85700b4500 2024-04-27T07:05:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f8574a95640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f8570146850 con 0x7f8570145ec0 2024-04-27T07:05:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f8574a95640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f85700b52b0 con 0x7f857013fef0 2024-04-27T07:05:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856ed9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8570145ec0 0x7f8570146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856ed9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8570145ec0 0x7f8570146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41194/0 (socket says 172.21.15.12:41194) 2024-04-27T07:05:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856ed9d640 1 -- 172.21.15.12:0/1722721664 learned_addr learned my addr 172.21.15.12:0/1722721664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f8567fff640 1 --2- 172.21.15.12:0/1722721664 >> v2:172.21.15.53:3300/0 conn(0x7f857013fef0 0x7f85700b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856ed9d640 1 -- 172.21.15.12:0/1722721664 >> v2:172.21.15.53:3300/0 conn(0x7f857013fef0 msgr2=0x7f85700b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856ed9d640 1 --2- 172.21.15.12:0/1722721664 >> v2:172.21.15.53:3300/0 conn(0x7f857013fef0 0x7f85700b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856ed9d640 1 -- 172.21.15.12:0/1722721664 >> v1:172.21.15.53:6789/0 conn(0x7f85700b4500 legacy=0x7f857013f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:25.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856ed9d640 1 -- 172.21.15.12:0/1722721664 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85700b7ae0 con 0x7f8570145ec0 2024-04-27T07:05:25.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f8567fff640 1 --2- 172.21.15.12:0/1722721664 >> v2:172.21.15.53:3300/0 conn(0x7f857013fef0 0x7f85700b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856ed9d640 1 --2- 172.21.15.12:0/1722721664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8570145ec0 0x7f8570146310 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8570074230 tx=0x7f856801c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638a4f9f0cbb0f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856dd9b640 1 -- 172.21.15.12:0/1722721664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85680ac020 con 0x7f8570145ec0 2024-04-27T07:05:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856dd9b640 1 -- 172.21.15.12:0/1722721664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85680a64a0 con 0x7f8570145ec0 2024-04-27T07:05:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.480+0000 7f856dd9b640 1 -- 172.21.15.12:0/1722721664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85680a6870 con 0x7f8570145ec0 2024-04-27T07:05:25.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- 172.21.15.12:0/1722721664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8570145ec0 msgr2=0x7f8570146310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 --2- 172.21.15.12:0/1722721664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8570145ec0 0x7f8570146310 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8570074230 tx=0x7f856801c7d0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- 172.21.15.12:0/1722721664 shutdown_connections 2024-04-27T07:05:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 --2- 172.21.15.12:0/1722721664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8570145ec0 0x7f8570146310 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 --2- 172.21.15.12:0/1722721664 >> v2:172.21.15.53:3300/0 conn(0x7f857013fef0 0x7f85700b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- 172.21.15.12:0/1722721664 >> 172.21.15.12:0/1722721664 conn(0x7f85700bb4f0 msgr2=0x7f85700bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- 172.21.15.12:0/1722721664 shutdown_connections 2024-04-27T07:05:25.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- 172.21.15.12:0/1722721664 wait complete. 2024-04-27T07:05:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 Processor -- start 2024-04-27T07:05:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- start start 2024-04-27T07:05:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b4500 0x7f857013a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f857013fef0 0x7f857006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8570145ec0 0x7f8570139070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8570152db0 con 0x7f857013fef0 2024-04-27T07:05:25.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f85700b2c00 con 0x7f85700b4500 2024-04-27T07:05:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8570152f30 con 0x7f8570145ec0 2024-04-27T07:05:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8567fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f857013fef0 0x7f857006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8567fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f857013fef0 0x7f857006cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41208/0 (socket says 172.21.15.12:41208) 2024-04-27T07:05:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8567fff640 1 -- 172.21.15.12:0/3677067127 learned_addr learned my addr 172.21.15.12:0/3677067127 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f856ed9d640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8570145ec0 0x7f8570139070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f856e59c640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b4500 0x7f857013a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8567fff640 1 -- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8570145ec0 msgr2=0x7f8570139070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8567fff640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8570145ec0 0x7f8570139070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8567fff640 1 -- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b4500 msgr2=0x7f857013a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8567fff640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b4500 0x7f857013a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8567fff640 1 -- 172.21.15.12:0/3677067127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85701357c0 con 0x7f857013fef0 2024-04-27T07:05:25.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f856ed9d640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8570145ec0 0x7f8570139070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:25.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f856e59c640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b4500 0x7f857013a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:25.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8567fff640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f857013fef0 0x7f857006cc90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f855800c7c0 tx=0x7f855800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:25.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8566ffd640 1 -- 172.21.15.12:0/3677067127 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558007a30 con 0x7f857013fef0 2024-04-27T07:05:25.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8566ffd640 1 -- 172.21.15.12:0/3677067127 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8558007bd0 con 0x7f857013fef0 2024-04-27T07:05:25.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8566ffd640 1 -- 172.21.15.12:0/3677067127 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85580113f0 con 0x7f857013fef0 2024-04-27T07:05:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- 172.21.15.12:0/3677067127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857013adb0 con 0x7f857013fef0 2024-04-27T07:05:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.484+0000 7f8574a95640 1 -- 172.21.15.12:0/3677067127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85701e3c00 con 0x7f857013fef0 2024-04-27T07:05:25.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.488+0000 7f8574a95640 1 -- 172.21.15.12:0/3677067127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8570069330 con 0x7f857013fef0 2024-04-27T07:05:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.488+0000 7f8566ffd640 1 -- 172.21.15.12:0/3677067127 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8558011590 con 0x7f857013fef0 2024-04-27T07:05:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.488+0000 7f8566ffd640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f85500683b0 0x7f855006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.488+0000 7f8566ffd640 1 -- 172.21.15.12:0/3677067127 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f85580868b0 con 0x7f857013fef0 2024-04-27T07:05:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.488+0000 7f856e59c640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f85500683b0 0x7f855006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.488+0000 7f856e59c640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f85500683b0 0x7f855006a870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8560003e40 tx=0x7f8560009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:25.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.492+0000 7f8566ffd640 1 -- 172.21.15.12:0/3677067127 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8558015020 con 0x7f857013fef0 2024-04-27T07:05:25.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.652+0000 7f8574a95640 1 -- 172.21.15.12:0/3677067127 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f857013abe0 con 0x7f85500683b0 2024-04-27T07:05:25.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8566ffd640 1 -- 172.21.15.12:0/3677067127 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7f856001b4c0 con 0x7f85500683b0 2024-04-27T07:05:25.661 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:25.661 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:05:25.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 -- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f85500683b0 msgr2=0x7f855006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f85500683b0 0x7f855006a870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8560003e40 tx=0x7f8560009040 comp rx=0 tx=0).stop 2024-04-27T07:05:25.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 -- 172.21.15.12:0/3677067127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f857013fef0 msgr2=0x7f857006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f857013fef0 0x7f857006cc90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f855800c7c0 tx=0x7f855800cc80 comp rx=0 tx=0).stop 2024-04-27T07:05:25.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 -- 172.21.15.12:0/3677067127 shutdown_connections 2024-04-27T07:05:25.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f85500683b0 0x7f855006a870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8570145ec0 0x7f8570139070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f857013fef0 0x7f857006cc90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 --2- 172.21.15.12:0/3677067127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85700b4500 0x7f857013a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 -- 172.21.15.12:0/3677067127 >> 172.21.15.12:0/3677067127 conn(0x7f85700bb4f0 msgr2=0x7f8570072560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:25.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 -- 172.21.15.12:0/3677067127 shutdown_connections 2024-04-27T07:05:25.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.656+0000 7f8574a95640 1 -- 172.21.15.12:0/3677067127 wait complete. 2024-04-27T07:05:25.680 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-27T07:05:24.632300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236840,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737880,"statfs":{"total":773094113280,"available":772851589120,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002132"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-27T07:05:19.386254+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:05:19.386254+0000","last_peered":"2024-04-27T07:05:19.386254+0000","last_clean":"2024-04-27T07:05:19.386254+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:05:19.386254+0000","last_undegraded":"2024-04-27T07:05:19.386254+0000","last_fullsized":"2024-04-27T07:05:19.386254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:05:25.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:05:25.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.800+0000 7f87d86be640 1 Processor -- start 2024-04-27T07:05:25.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.800+0000 7f87d86be640 1 -- start start 2024-04-27T07:05:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d86be640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f87d00b9a10 0x7f87d00b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d86be640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f87d014e400 0x7f87d01507f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d86be640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d006dac0 con 0x7f87d00b3680 2024-04-27T07:05:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d86be640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f87d006e890 con 0x7f87d014e400 2024-04-27T07:05:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d86be640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f87d00b9df0 con 0x7f87d00b9a10 2024-04-27T07:05:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d6c5b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f87d014e400 0x7f87d01507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d5c59640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f87d00b3680 0x7f87d0145b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57984/0 (socket says 172.21.15.12:57984) 2024-04-27T07:05:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d5c59640 1 -- 172.21.15.12:0/1991916059 learned_addr learned my addr 172.21.15.12:0/1991916059 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d645a640 1 --2- 172.21.15.12:0/1991916059 >> v2:172.21.15.53:3300/0 conn(0x7f87d00b9a10 0x7f87d00b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d6c5b640 1 -- 172.21.15.12:0/1991916059 >> v2:172.21.15.53:3300/0 conn(0x7f87d00b9a10 msgr2=0x7f87d00b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:25.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d6c5b640 1 --2- 172.21.15.12:0/1991916059 >> v2:172.21.15.53:3300/0 conn(0x7f87d00b9a10 0x7f87d00b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d6c5b640 1 -- 172.21.15.12:0/1991916059 >> v1:172.21.15.12:6789/0 conn(0x7f87d00b3680 legacy=0x7f87d0145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d6c5b640 1 -- 172.21.15.12:0/1991916059 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d006e090 con 0x7f87d014e400 2024-04-27T07:05:25.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d6c5b640 1 --2- 172.21.15.12:0/1991916059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 0x7f87d01507f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f87d006ed00 tx=0x7f87c801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1915a50353dce74 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d5458640 1 -- 172.21.15.12:0/1991916059 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c80b2070 con 0x7f87d014e400 2024-04-27T07:05:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d5458640 1 -- 172.21.15.12:0/1991916059 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c80a5760 con 0x7f87d014e400 2024-04-27T07:05:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.804+0000 7f87d5458640 1 -- 172.21.15.12:0/1991916059 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c80a5b30 con 0x7f87d014e400 2024-04-27T07:05:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- 172.21.15.12:0/1991916059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 msgr2=0x7f87d01507f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 --2- 172.21.15.12:0/1991916059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 0x7f87d01507f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f87d006ed00 tx=0x7f87c801c8a0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- 172.21.15.12:0/1991916059 shutdown_connections 2024-04-27T07:05:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 --2- 172.21.15.12:0/1991916059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 0x7f87d01507f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 --2- 172.21.15.12:0/1991916059 >> v2:172.21.15.53:3300/0 conn(0x7f87d00b9a10 0x7f87d00b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- 172.21.15.12:0/1991916059 >> 172.21.15.12:0/1991916059 conn(0x7f87d0067d10 msgr2=0x7f87d0068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- 172.21.15.12:0/1991916059 shutdown_connections 2024-04-27T07:05:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- 172.21.15.12:0/1991916059 wait complete. 2024-04-27T07:05:25.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 Processor -- start 2024-04-27T07:05:25.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- start start 2024-04-27T07:05:25.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87d00b3680 0x7f87d009d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87d00b9a10 0x7f87d0072b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 0x7f87d01e1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0074020 con 0x7f87d014e400 2024-04-27T07:05:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0073ea0 con 0x7f87d00b9a10 2024-04-27T07:05:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f87d00b6b00 con 0x7f87d00b3680 2024-04-27T07:05:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d6c5b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 0x7f87d01e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d6c5b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 0x7f87d01e1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41224/0 (socket says 172.21.15.12:41224) 2024-04-27T07:05:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d6c5b640 1 -- 172.21.15.12:0/503579069 learned_addr learned my addr 172.21.15.12:0/503579069 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d5c59640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87d00b9a10 0x7f87d0072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d645a640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87d00b3680 0x7f87d009d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d6c5b640 1 -- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87d00b3680 msgr2=0x7f87d009d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d6c5b640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87d00b3680 0x7f87d009d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d6c5b640 1 -- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87d00b9a10 msgr2=0x7f87d0072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d6c5b640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87d00b9a10 0x7f87d0072b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d6c5b640 1 -- 172.21.15.12:0/503579069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d0073050 con 0x7f87d014e400 2024-04-27T07:05:25.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d6c5b640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 0x7f87d01e1750 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f87c801a4d0 tx=0x7f87c8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:25.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87c6ffd640 1 -- 172.21.15.12:0/503579069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c80b2050 con 0x7f87d014e400 2024-04-27T07:05:25.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87c6ffd640 1 -- 172.21.15.12:0/503579069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c80be070 con 0x7f87d014e400 2024-04-27T07:05:25.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- 172.21.15.12:0/503579069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d0071fa0 con 0x7f87d014e400 2024-04-27T07:05:25.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87c6ffd640 1 -- 172.21.15.12:0/503579069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c80ad8b0 con 0x7f87d014e400 2024-04-27T07:05:25.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.808+0000 7f87d86be640 1 -- 172.21.15.12:0/503579069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d00725f0 con 0x7f87d014e400 2024-04-27T07:05:25.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.812+0000 7f87d86be640 1 -- 172.21.15.12:0/503579069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d00709b0 con 0x7f87d014e400 2024-04-27T07:05:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.812+0000 7f87c6ffd640 1 -- 172.21.15.12:0/503579069 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f87c80ada50 con 0x7f87d014e400 2024-04-27T07:05:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.812+0000 7f87c6ffd640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f87b40683b0 0x7f87b406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.812+0000 7f87c6ffd640 1 -- 172.21.15.12:0/503579069 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f87c81212f0 con 0x7f87d014e400 2024-04-27T07:05:25.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.812+0000 7f87d645a640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f87b40683b0 0x7f87b406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:25.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.812+0000 7f87d645a640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f87b40683b0 0x7f87b406a870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009790 tx=0x7f87c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:25.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.816+0000 7f87c6ffd640 1 -- 172.21.15.12:0/503579069 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f87c80c7020 con 0x7f87d014e400 2024-04-27T07:05:25.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.972+0000 7f87d86be640 1 -- 172.21.15.12:0/503579069 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87d0068c20 con 0x7f87b40683b0 2024-04-27T07:05:25.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87c6ffd640 1 -- 172.21.15.12:0/503579069 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7f87d0068c20 con 0x7f87b40683b0 2024-04-27T07:05:25.982 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:25.983 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:05:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 -- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f87b40683b0 msgr2=0x7f87b406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f87b40683b0 0x7f87b406a870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009790 tx=0x7f87c0008040 comp rx=0 tx=0).stop 2024-04-27T07:05:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 -- 172.21.15.12:0/503579069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 msgr2=0x7f87d01e1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 0x7f87d01e1750 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f87c801a4d0 tx=0x7f87c8019640 comp rx=0 tx=0).stop 2024-04-27T07:05:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 -- 172.21.15.12:0/503579069 shutdown_connections 2024-04-27T07:05:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f87b40683b0 0x7f87b406a870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87d014e400 0x7f87d01e1750 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87d00b9a10 0x7f87d0072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 --2- 172.21.15.12:0/503579069 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87d00b3680 0x7f87d009d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 -- 172.21.15.12:0/503579069 >> 172.21.15.12:0/503579069 conn(0x7f87d0067d10 msgr2=0x7f87d009b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 -- 172.21.15.12:0/503579069 shutdown_connections 2024-04-27T07:05:25.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:25.976+0000 7f87d86be640 1 -- 172.21.15.12:0/503579069 wait complete. 2024-04-27T07:05:26.001 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-27T07:05:24.632300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236840,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737880,"statfs":{"total":773094113280,"available":772851589120,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002132"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-27T07:05:19.386254+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:05:19.386254+0000","last_peered":"2024-04-27T07:05:19.386254+0000","last_clean":"2024-04-27T07:05:19.386254+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:05:19.386254+0000","last_undegraded":"2024-04-27T07:05:19.386254+0000","last_fullsized":"2024-04-27T07:05:19.386254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:05:26.001 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-27T07:05:26.002 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:05:26.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.120+0000 7f9f6d0b8640 1 Processor -- start 2024-04-27T07:05:26.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.120+0000 7f9f6d0b8640 1 -- start start 2024-04-27T07:05:26.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6d0b8640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f9f6806e6b0 0x7f9f681418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6d0b8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9f6806cc90 0x7f9f681481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6d0b8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f6806ea90 con 0x7f9f68141e00 2024-04-27T07:05:26.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6d0b8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f9f6806df30 con 0x7f9f6806cc90 2024-04-27T07:05:26.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6d0b8640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f9f68147cc0 con 0x7f9f6806e6b0 2024-04-27T07:05:26.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9f6806cc90 0x7f9f681481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9f6806cc90 0x7f9f681481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41240/0 (socket says 172.21.15.12:41240) 2024-04-27T07:05:26.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6659c640 1 -- 172.21.15.12:0/371576091 learned_addr learned my addr 172.21.15.12:0/371576091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:26.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6759e640 1 --1- 172.21.15.12:0/371576091 >> v1:172.21.15.12:6789/0 conn(0x7f9f68141e00 0x7f9f6814b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57990/0 (socket says 172.21.15.12:57990) 2024-04-27T07:05:26.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f66d9d640 1 --2- 172.21.15.12:0/371576091 >> v2:172.21.15.53:3300/0 conn(0x7f9f6806e6b0 0x7f9f681418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6659c640 1 -- 172.21.15.12:0/371576091 >> v2:172.21.15.53:3300/0 conn(0x7f9f6806e6b0 msgr2=0x7f9f681418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:26.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6659c640 1 --2- 172.21.15.12:0/371576091 >> v2:172.21.15.53:3300/0 conn(0x7f9f6806e6b0 0x7f9f681418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6659c640 1 -- 172.21.15.12:0/371576091 >> v1:172.21.15.12:6789/0 conn(0x7f9f68141e00 legacy=0x7f9f6814b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6659c640 1 -- 172.21.15.12:0/371576091 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f680b23b0 con 0x7f9f6806cc90 2024-04-27T07:05:26.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f6659c640 1 --2- 172.21.15.12:0/371576091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 0x7f9f681481c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9f6806f1a0 tx=0x7f9f5001d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e64614886e2136 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:26.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f65d9b640 1 -- 172.21.15.12:0/371576091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f500a5730 con 0x7f9f6806cc90 2024-04-27T07:05:26.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f65d9b640 1 -- 172.21.15.12:0/371576091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f500a58d0 con 0x7f9f6806cc90 2024-04-27T07:05:26.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.124+0000 7f9f65d9b640 1 -- 172.21.15.12:0/371576091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f500a5ca0 con 0x7f9f6806cc90 2024-04-27T07:05:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/371576091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 msgr2=0x7f9f681481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 --2- 172.21.15.12:0/371576091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 0x7f9f681481c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9f6806f1a0 tx=0x7f9f5001d3c0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/371576091 shutdown_connections 2024-04-27T07:05:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 --2- 172.21.15.12:0/371576091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 0x7f9f681481c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 --2- 172.21.15.12:0/371576091 >> v2:172.21.15.53:3300/0 conn(0x7f9f6806e6b0 0x7f9f681418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/371576091 >> 172.21.15.12:0/371576091 conn(0x7f9f680bb4f0 msgr2=0x7f9f680bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/371576091 shutdown_connections 2024-04-27T07:05:26.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/371576091 wait complete. 2024-04-27T07:05:26.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 Processor -- start 2024-04-27T07:05:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- start start 2024-04-27T07:05:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 0x7f9f6813d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6806e6b0 0x7f9f681e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f68141e00 0x7f9f6813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681556d0 con 0x7f9f6806cc90 2024-04-27T07:05:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68155550 con 0x7f9f6806e6b0 2024-04-27T07:05:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f9f68155850 con 0x7f9f68141e00 2024-04-27T07:05:26.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f66d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 0x7f9f6813d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f66d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 0x7f9f6813d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41248/0 (socket says 172.21.15.12:41248) 2024-04-27T07:05:26.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f66d9d640 1 -- 172.21.15.12:0/2586473694 learned_addr learned my addr 172.21.15.12:0/2586473694 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:26.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6659c640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6806e6b0 0x7f9f681e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6759e640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f68141e00 0x7f9f6813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f66d9d640 1 -- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f68141e00 msgr2=0x7f9f6813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f66d9d640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f68141e00 0x7f9f6813f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f66d9d640 1 -- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6806e6b0 msgr2=0x7f9f681e3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f66d9d640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6806e6b0 0x7f9f681e3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f66d9d640 1 -- 172.21.15.12:0/2586473694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f680b40f0 con 0x7f9f6806cc90 2024-04-27T07:05:26.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6659c640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6806e6b0 0x7f9f681e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:26.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f66d9d640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 0x7f9f6813d4d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800d830 tx=0x7f9f5800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:26.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f5f7fe640 1 -- 172.21.15.12:0/2586473694 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800b6c0 con 0x7f9f6806cc90 2024-04-27T07:05:26.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f5f7fe640 1 -- 172.21.15.12:0/2586473694 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f58019070 con 0x7f9f6806cc90 2024-04-27T07:05:26.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/2586473694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f680b4f50 con 0x7f9f6806cc90 2024-04-27T07:05:26.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f5f7fe640 1 -- 172.21.15.12:0/2586473694 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800ba60 con 0x7f9f6806cc90 2024-04-27T07:05:26.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.128+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/2586473694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f68140490 con 0x7f9f6806cc90 2024-04-27T07:05:26.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.132+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/2586473694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f68069330 con 0x7f9f6806cc90 2024-04-27T07:05:26.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.132+0000 7f9f5f7fe640 1 -- 172.21.15.12:0/2586473694 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9f5801e460 con 0x7f9f6806cc90 2024-04-27T07:05:26.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.132+0000 7f9f5f7fe640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9f480683b0 0x7f9f4806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.132+0000 7f9f5f7fe640 1 -- 172.21.15.12:0/2586473694 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f9f58086cf0 con 0x7f9f6806cc90 2024-04-27T07:05:26.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.132+0000 7f9f6659c640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9f480683b0 0x7f9f4806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.132+0000 7f9f6659c640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9f480683b0 0x7f9f4806a870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9f500ae000 tx=0x7f9f50004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:26.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.136+0000 7f9f5f7fe640 1 -- 172.21.15.12:0/2586473694 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9f5808b070 con 0x7f9f6806cc90 2024-04-27T07:05:26.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.292+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/2586473694 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f68146c10 con 0x7f9f480683b0 2024-04-27T07:05:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f5f7fe640 1 -- 172.21.15.12:0/2586473694 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7f9f68146c10 con 0x7f9f480683b0 2024-04-27T07:05:26.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:05:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9f480683b0 msgr2=0x7f9f4806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9f480683b0 0x7f9f4806a870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9f500ae000 tx=0x7f9f50004ac0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/2586473694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 msgr2=0x7f9f6813d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 0x7f9f6813d4d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800d830 tx=0x7f9f5800dcf0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/2586473694 shutdown_connections 2024-04-27T07:05:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f9f480683b0 0x7f9f4806a870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f9f68141e00 0x7f9f6813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9f6806e6b0 0x7f9f681e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 --2- 172.21.15.12:0/2586473694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f6806cc90 0x7f9f6813d4d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/2586473694 >> 172.21.15.12:0/2586473694 conn(0x7f9f680bb4f0 msgr2=0x7f9f68072560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:26.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.296+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/2586473694 shutdown_connections 2024-04-27T07:05:26.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.300+0000 7f9f6d0b8640 1 -- 172.21.15.12:0/2586473694 wait complete. 2024-04-27T07:05:26.320 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-27T07:05:24.632300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236840,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737880,"statfs":{"total":773094113280,"available":772851589120,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002132"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-27T07:05:19.386254+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:05:19.386254+0000","last_peered":"2024-04-27T07:05:19.386254+0000","last_clean":"2024-04-27T07:05:19.386254+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:05:19.386254+0000","last_undegraded":"2024-04-27T07:05:19.386254+0000","last_fullsized":"2024-04-27T07:05:19.386254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:05:26.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T07:05:26.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.436+0000 7f7b5eb55640 1 Processor -- start 2024-04-27T07:05:26.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.436+0000 7f7b5eb55640 1 -- start start 2024-04-27T07:05:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b580b62a0 0x7f7b580b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f7b581485b0 0x7f7b5814b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b58148a30 0x7f7b58141830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f7b58147db0 con 0x7f7b58148a30 2024-04-27T07:05:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f7b58148090 con 0x7f7b581485b0 2024-04-27T07:05:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5814c170 con 0x7f7b580b62a0 2024-04-27T07:05:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5d0f2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b58148a30 0x7f7b58141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5d0f2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b58148a30 0x7f7b58141830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41254/0 (socket says 172.21.15.12:41254) 2024-04-27T07:05:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5d0f2640 1 -- 172.21.15.12:0/210651515 learned_addr learned my addr 172.21.15.12:0/210651515 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5c8f1640 1 --2- 172.21.15.12:0/210651515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b580b62a0 0x7f7b580b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b57fff640 1 --2- 172.21.15.12:0/210651515 >> v2:172.21.15.53:3300/0 conn(0x7f7b581485b0 0x7f7b5814b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5d0f2640 1 -- 172.21.15.12:0/210651515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b580b62a0 msgr2=0x7f7b580b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5d0f2640 1 --2- 172.21.15.12:0/210651515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b580b62a0 0x7f7b580b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5d0f2640 1 -- 172.21.15.12:0/210651515 >> v2:172.21.15.53:3300/0 conn(0x7f7b581485b0 msgr2=0x7f7b5814b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5d0f2640 1 --2- 172.21.15.12:0/210651515 >> v2:172.21.15.53:3300/0 conn(0x7f7b581485b0 0x7f7b5814b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5d0f2640 1 -- 172.21.15.12:0/210651515 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b580b7f60 con 0x7f7b58148a30 2024-04-27T07:05:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5d0f2640 1 --2- 172.21.15.12:0/210651515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b58148a30 0x7f7b58141830 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7b580bb070 tx=0x7f7b5001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd151dd0f6606726 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b577fe640 1 -- 172.21.15.12:0/210651515 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b500a56f0 con 0x7f7b58148a30 2024-04-27T07:05:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b577fe640 1 -- 172.21.15.12:0/210651515 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b500a5890 con 0x7f7b58148a30 2024-04-27T07:05:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b577fe640 1 -- 172.21.15.12:0/210651515 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b500a5c60 con 0x7f7b58148a30 2024-04-27T07:05:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 -- 172.21.15.12:0/210651515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b58148a30 msgr2=0x7f7b58141830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 --2- 172.21.15.12:0/210651515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b58148a30 0x7f7b58141830 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7b580bb070 tx=0x7f7b5001c960 comp rx=0 tx=0).stop 2024-04-27T07:05:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 -- 172.21.15.12:0/210651515 shutdown_connections 2024-04-27T07:05:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 --2- 172.21.15.12:0/210651515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b58148a30 0x7f7b58141830 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 --2- 172.21.15.12:0/210651515 >> v2:172.21.15.53:3300/0 conn(0x7f7b581485b0 0x7f7b5814b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 --2- 172.21.15.12:0/210651515 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b580b62a0 0x7f7b580b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 -- 172.21.15.12:0/210651515 >> 172.21.15.12:0/210651515 conn(0x7f7b58068190 msgr2=0x7f7b580bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 -- 172.21.15.12:0/210651515 shutdown_connections 2024-04-27T07:05:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.440+0000 7f7b5eb55640 1 -- 172.21.15.12:0/210651515 wait complete. 2024-04-27T07:05:26.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 Processor -- start 2024-04-27T07:05:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 -- start start 2024-04-27T07:05:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b580b62a0 0x7f7b581e3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b581485b0 0x7f7b5813add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b58148a30 0x7f7b58137f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b580b5b20 con 0x7f7b581485b0 2024-04-27T07:05:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b580b59a0 con 0x7f7b580b62a0 2024-04-27T07:05:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f7b580b5ca0 con 0x7f7b58148a30 2024-04-27T07:05:26.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b57fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b581485b0 0x7f7b5813add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b57fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b581485b0 0x7f7b5813add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41256/0 (socket says 172.21.15.12:41256) 2024-04-27T07:05:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b57fff640 1 -- 172.21.15.12:0/657975365 learned_addr learned my addr 172.21.15.12:0/657975365 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5d0f2640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b58148a30 0x7f7b58137f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5c8f1640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b580b62a0 0x7f7b581e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b57fff640 1 -- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b58148a30 msgr2=0x7f7b58137f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b57fff640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b58148a30 0x7f7b58137f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b57fff640 1 -- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b580b62a0 msgr2=0x7f7b581e3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b57fff640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b580b62a0 0x7f7b581e3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b57fff640 1 -- 172.21.15.12:0/657975365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b58137a90 con 0x7f7b581485b0 2024-04-27T07:05:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b57fff640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b581485b0 0x7f7b5813add0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000d5f0 tx=0x7f7b4000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b557fa640 1 -- 172.21.15.12:0/657975365 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b400125f0 con 0x7f7b581485b0 2024-04-27T07:05:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b557fa640 1 -- 172.21.15.12:0/657975365 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b40019070 con 0x7f7b581485b0 2024-04-27T07:05:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 -- 172.21.15.12:0/657975365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b581368c0 con 0x7f7b581485b0 2024-04-27T07:05:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b557fa640 1 -- 172.21.15.12:0/657975365 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b400129b0 con 0x7f7b581485b0 2024-04-27T07:05:26.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 -- 172.21.15.12:0/657975365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b581377b0 con 0x7f7b581485b0 2024-04-27T07:05:26.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.444+0000 7f7b5eb55640 1 -- 172.21.15.12:0/657975365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b58069330 con 0x7f7b581485b0 2024-04-27T07:05:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.448+0000 7f7b557fa640 1 -- 172.21.15.12:0/657975365 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7b40012b50 con 0x7f7b581485b0 2024-04-27T07:05:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.448+0000 7f7b557fa640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7b38068380 0x7f7b3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.448+0000 7f7b557fa640 1 -- 172.21.15.12:0/657975365 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f7b40087430 con 0x7f7b581485b0 2024-04-27T07:05:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.448+0000 7f7b5c8f1640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7b38068380 0x7f7b3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:26.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.448+0000 7f7b5c8f1640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7b38068380 0x7f7b3806a840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800b930 tx=0x7f7b48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:26.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.452+0000 7f7b557fa640 1 -- 172.21.15.12:0/657975365 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7b4001c360 con 0x7f7b581485b0 2024-04-27T07:05:26.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.612+0000 7f7b5eb55640 1 -- 172.21.15.12:0/657975365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b581393f0 con 0x7f7b581485b0 2024-04-27T07:05:26.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.612+0000 7f7b557fa640 1 -- 172.21.15.12:0/657975365 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+12960 (secure 0 0 0) 0x7f7b40085e20 con 0x7f7b581485b0 2024-04-27T07:05:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:26.620 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"fsid":"e023690c-eecf-4ce8-af7e-ab331203966f","created":"2024-04-27T07:03:57.714342+0000","modified":"2024-04-27T07:05:19.053325+0000","last_up_change":"2024-04-27T07:04:08.349154+0000","last_in_change":"2024-04-27T07:04:00.046535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T07:04:08.650986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57fa2165-d30f-4ee5-a2c8-ad17c3948dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6817","nonce":3217679893}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6823","nonce":3217679893}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6831","nonce":3217679893}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3217679893},{"type":"v1","addr":"172.21.15.12:6828","nonce":3217679893}]},"public_addr":"172.21.15.12:6817/3217679893","cluster_addr":"172.21.15.12:6823/3217679893","heartbeat_back_addr":"172.21.15.12:6831/3217679893","heartbeat_front_addr":"172.21.15.12:6828/3217679893","state":["exists","up"]},{"osd":1,"uuid":"7bdf73e9-6350-46e3-bbfb-45f0ca6776d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6801","nonce":3260695366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6803","nonce":3260695366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6807","nonce":3260695366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":3260695366},{"type":"v1","addr":"172.21.15.12:6805","nonce":3260695366}]},"public_addr":"172.21.15.12:6801/3260695366","cluster_addr":"172.21.15.12:6803/3260695366","heartbeat_back_addr":"172.21.15.12:6807/3260695366","heartbeat_front_addr":"172.21.15.12:6805/3260695366","state":["exists","up"]},{"osd":2,"uuid":"6546def3-1651-4b63-83df-7cd53215ab49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6811","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6815","nonce":3457925485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6819","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6822","nonce":3457925485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6827","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6829","nonce":3457925485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3457925485},{"type":"v1","addr":"172.21.15.12:6825","nonce":3457925485}]},"public_addr":"172.21.15.12:6815/3457925485","cluster_addr":"172.21.15.12:6822/3457925485","heartbeat_back_addr":"172.21.15.12:6829/3457925485","heartbeat_front_addr":"172.21.15.12:6825/3457925485","state":["exists","up"]},{"osd":3,"uuid":"ec12ea7e-5e96-4e37-b733-d997a7f2cb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6809","nonce":2279477907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6812","nonce":2279477907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6820","nonce":2279477907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":2279477907},{"type":"v1","addr":"172.21.15.12:6816","nonce":2279477907}]},"public_addr":"172.21.15.12:6809/2279477907","cluster_addr":"172.21.15.12:6812/2279477907","heartbeat_back_addr":"172.21.15.12:6820/2279477907","heartbeat_front_addr":"172.21.15.12:6816/2279477907","state":["exists","up"]},{"osd":4,"uuid":"4fb26f06-cac3-4c15-a77e-2826ef9c9781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6801","nonce":151807054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6803","nonce":151807054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6807","nonce":151807054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":151807054},{"type":"v1","addr":"172.21.15.53:6805","nonce":151807054}]},"public_addr":"172.21.15.53:6801/151807054","cluster_addr":"172.21.15.53:6803/151807054","heartbeat_back_addr":"172.21.15.53:6807/151807054","heartbeat_front_addr":"172.21.15.53:6805/151807054","state":["exists","up"]},{"osd":5,"uuid":"13c23066-aafd-48ba-894a-31b6ba9f76ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6825","nonce":2061790692}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6827","nonce":2061790692}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6831","nonce":2061790692}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2061790692},{"type":"v1","addr":"172.21.15.53:6829","nonce":2061790692}]},"public_addr":"172.21.15.53:6825/2061790692","cluster_addr":"172.21.15.53:6827/2061790692","heartbeat_back_addr":"172.21.15.53:6831/2061790692","heartbeat_front_addr":"172.21.15.53:6829/2061790692","state":["exists","up"]},{"osd":6,"uuid":"5f6a8c98-d79d-4d26-b42b-d2f0b332b951","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6809","nonce":289427539}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6811","nonce":289427539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6815","nonce":289427539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":289427539},{"type":"v1","addr":"172.21.15.53:6813","nonce":289427539}]},"public_addr":"172.21.15.53:6809/289427539","cluster_addr":"172.21.15.53:6811/289427539","heartbeat_back_addr":"172.21.15.53:6815/289427539","heartbeat_front_addr":"172.21.15.53:6813/289427539","state":["exists","up"]},{"osd":7,"uuid":"4792d49e-3f64-40db-ba32-c1975da10ee5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6817","nonce":2338388162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6819","nonce":2338388162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6823","nonce":2338388162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2338388162},{"type":"v1","addr":"172.21.15.53:6821","nonce":2338388162}]},"public_addr":"172.21.15.53:6817/2338388162","cluster_addr":"172.21.15.53:6819/2338388162","heartbeat_back_addr":"172.21.15.53:6823/2338388162","heartbeat_front_addr":"172.21.15.53:6821/2338388162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.583170+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.539160+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.660563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.633063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.629612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.359108+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.584135+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T07:04:05.448336+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6833/4067812811":"2024-04-28T07:05:16.038378+0000","172.21.15.12:6832/4067812811":"2024-04-28T07:05:16.038378+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T07:05:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 -- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7b38068380 msgr2=0x7f7b3806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7b38068380 0x7f7b3806a840 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800b930 tx=0x7f7b48008040 comp rx=0 tx=0).stop 2024-04-27T07:05:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 -- 172.21.15.12:0/657975365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b581485b0 msgr2=0x7f7b5813add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b581485b0 0x7f7b5813add0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000d5f0 tx=0x7f7b4000dab0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 -- 172.21.15.12:0/657975365 shutdown_connections 2024-04-27T07:05:26.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f7b38068380 0x7f7b3806a840 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f7b58148a30 0x7f7b58137f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b581485b0 0x7f7b5813add0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 --2- 172.21.15.12:0/657975365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b580b62a0 0x7f7b581e3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:26.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 -- 172.21.15.12:0/657975365 >> 172.21.15.12:0/657975365 conn(0x7f7b58068190 msgr2=0x7f7b58074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:26.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 -- 172.21.15.12:0/657975365 shutdown_connections 2024-04-27T07:05:26.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:26.616+0000 7f7b5eb55640 1 -- 172.21.15.12:0/657975365 wait complete. 2024-04-27T07:05:27.640 INFO:tasks.ceph:Scrubbing osd.0 2024-04-27T07:05:27.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:05:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff19873e640 1 Processor -- start 2024-04-27T07:05:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff19873e640 1 -- start start 2024-04-27T07:05:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff19873e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff1900ad920 0x7ff19009da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff19873e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff19006c9a0 con 0x7ff19009df70 2024-04-27T07:05:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff19873e640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1900add00 con 0x7ff1900b4df0 2024-04-27T07:05:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff19873e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7ff19009d5a0 con 0x7ff1900ad920 2024-04-27T07:05:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff196cdb640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff19009df70 0x7ff19006c290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52210/0 (socket says 172.21.15.12:52210) 2024-04-27T07:05:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff196cdb640 1 -- 172.21.15.12:0/2863732010 learned_addr learned my addr 172.21.15.12:0/2863732010 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff1964da640 1 --2- 172.21.15.12:0/2863732010 >> v2:172.21.15.12:3300/0 conn(0x7ff1900ad920 0x7ff19009da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff1954d8640 1 -- 172.21.15.12:0/2863732010 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979727601 0 0) 0x7ff19006c9a0 con 0x7ff19009df70 2024-04-27T07:05:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff1954d8640 1 -- 172.21.15.12:0/2863732010 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff17c003530 con 0x7ff19009df70 2024-04-27T07:05:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff1964da640 1 -- 172.21.15.12:0/2863732010 >> v1:172.21.15.53:6789/0 conn(0x7ff1900b4df0 legacy=0x7ff19006bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff1964da640 1 -- 172.21.15.12:0/2863732010 >> v1:172.21.15.12:6789/0 conn(0x7ff19009df70 legacy=0x7ff19006c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.776+0000 7ff1964da640 1 -- 172.21.15.12:0/2863732010 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1900b5310 con 0x7ff1900ad920 2024-04-27T07:05:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff1964da640 1 --2- 172.21.15.12:0/2863732010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1900ad920 0x7ff19009da30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff1840097c0 tx=0x7ff18401c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2aec84eb7096a60 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff1954d8640 1 -- 172.21.15.12:0/2863732010 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1840b0070 con 0x7ff1900ad920 2024-04-27T07:05:27.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff1954d8640 1 -- 172.21.15.12:0/2863732010 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1840ab600 con 0x7ff1900ad920 2024-04-27T07:05:27.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff1954d8640 1 -- 172.21.15.12:0/2863732010 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1840ab9b0 con 0x7ff1900ad920 2024-04-27T07:05:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 -- 172.21.15.12:0/2863732010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1900ad920 msgr2=0x7ff19009da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 --2- 172.21.15.12:0/2863732010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1900ad920 0x7ff19009da30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff1840097c0 tx=0x7ff18401c350 comp rx=0 tx=0).stop 2024-04-27T07:05:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 -- 172.21.15.12:0/2863732010 shutdown_connections 2024-04-27T07:05:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 --2- 172.21.15.12:0/2863732010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1900ad920 0x7ff19009da30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 -- 172.21.15.12:0/2863732010 >> 172.21.15.12:0/2863732010 conn(0x7ff19006d550 msgr2=0x7ff1900ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 -- 172.21.15.12:0/2863732010 shutdown_connections 2024-04-27T07:05:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 -- 172.21.15.12:0/2863732010 wait complete. 2024-04-27T07:05:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 Processor -- start 2024-04-27T07:05:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 -- start start 2024-04-27T07:05:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff19009df70 0x7ff190150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1900ad920 0x7ff1901e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1900b4df0 0x7ff1901e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff19006e830 con 0x7ff1900b4df0 2024-04-27T07:05:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff1900b5df0 con 0x7ff1900ad920 2024-04-27T07:05:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff19873e640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7ff19006e9b0 con 0x7ff19009df70 2024-04-27T07:05:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff196cdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1900b4df0 0x7ff1901e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff196cdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1900b4df0 0x7ff1901e4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55532/0 (socket says 172.21.15.12:55532) 2024-04-27T07:05:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.780+0000 7ff196cdb640 1 -- 172.21.15.12:0/1684961997 learned_addr learned my addr 172.21.15.12:0/1684961997 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff1964da640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff19009df70 0x7ff190150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff195cd9640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1900ad920 0x7ff1901e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff196cdb640 1 -- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff19009df70 msgr2=0x7ff190150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff196cdb640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7ff19009df70 0x7ff190150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff196cdb640 1 -- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1900ad920 msgr2=0x7ff1901e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff196cdb640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1900ad920 0x7ff1901e49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff196cdb640 1 -- 172.21.15.12:0/1684961997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff190152fd0 con 0x7ff1900b4df0 2024-04-27T07:05:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff196cdb640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1900b4df0 0x7ff1901e4ee0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff18800c480 tx=0x7ff18800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff182ffd640 1 -- 172.21.15.12:0/1684961997 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188017070 con 0x7ff1900b4df0 2024-04-27T07:05:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff182ffd640 1 -- 172.21.15.12:0/1684961997 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff188005cf0 con 0x7ff1900b4df0 2024-04-27T07:05:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff182ffd640 1 -- 172.21.15.12:0/1684961997 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1880072b0 con 0x7ff1900b4df0 2024-04-27T07:05:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1900b3f80 con 0x7ff1900b4df0 2024-04-27T07:05:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1901532b0 con 0x7ff1900b4df0 2024-04-27T07:05:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff190069330 con 0x7ff1900b4df0 2024-04-27T07:05:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff182ffd640 1 -- 172.21.15.12:0/1684961997 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff18801a070 con 0x7ff1900b4df0 2024-04-27T07:05:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff182ffd640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff174068380 0x7ff17406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff182ffd640 1 -- 172.21.15.12:0/1684961997 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7ff188085990 con 0x7ff1900b4df0 2024-04-27T07:05:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff182ffd640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] conn(0x7ff17406e240 0x7ff174070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff182ffd640 1 -- 172.21.15.12:0/1684961997 --> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff18801be50 con 0x7ff17406e240 2024-04-27T07:05:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff182ffd640 1 -- 172.21.15.12:0/1684961997 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff18808a2c0 con 0x7ff1900b4df0 2024-04-27T07:05:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff195cd9640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] conn(0x7ff17406e240 0x7ff174070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff1964da640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff174068380 0x7ff17406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff195cd9640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] conn(0x7ff17406e240 0x7ff174070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.784+0000 7ff1964da640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff174068380 0x7ff17406a840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff1840ad000 tx=0x7ff1840a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.788+0000 7ff182ffd640 1 -- 172.21.15.12:0/1684961997 <== osd.0 v2:172.21.15.12:6813/3217679893 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff18801be50 con 0x7ff17406e240 2024-04-27T07:05:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 --> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff19014ff40 con 0x7ff17406e240 2024-04-27T07:05:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff182ffd640 1 -- 172.21.15.12:0/1684961997 <== osd.0 v2:172.21.15.12:6813/3217679893 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff19014ff40 con 0x7ff17406e240 2024-04-27T07:05:27.806 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:05:27.806 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:05:27.806 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:05:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 >> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] conn(0x7ff17406e240 msgr2=0x7ff174070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.12:6813/3217679893,v1:172.21.15.12:6817/3217679893] conn(0x7ff17406e240 0x7ff174070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff174068380 msgr2=0x7ff17406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7ff174068380 0x7ff17406a840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff1840ad000 tx=0x7ff1840a6000 comp rx=0 tx=0).stop 2024-04-27T07:05:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1900b4df0 msgr2=0x7ff1901e4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 --2- 172.21.15.12:0/1684961997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1900b4df0 0x7ff1901e4ee0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff18800c480 tx=0x7ff18800c940 comp rx=0 tx=0).stop 2024-04-27T07:05:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff196cdb640 1 -- 172.21.15.12:0/1684961997 reap_dead start 2024-04-27T07:05:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 shutdown_connections 2024-04-27T07:05:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 >> 172.21.15.12:0/1684961997 conn(0x7ff19006d550 msgr2=0x7ff19006c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 shutdown_connections 2024-04-27T07:05:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.800+0000 7ff19873e640 1 -- 172.21.15.12:0/1684961997 wait complete. 2024-04-27T07:05:27.822 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-27T07:05:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f015eaed640 1 Processor -- start 2024-04-27T07:05:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f015eaed640 1 -- start start 2024-04-27T07:05:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f015eaed640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f01580b6480 0x7f01580b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f015eaed640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f01581489c0 0x7f01580b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f015eaed640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0158141900 0x7f0158141d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f015eaed640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f01580b84e0 con 0x7f01581489c0 2024-04-27T07:05:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f015eaed640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f01580b6860 con 0x7f01580b6480 2024-04-27T07:05:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f015eaed640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f0158147ee0 con 0x7f0158141900 2024-04-27T07:05:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f0157fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f01581489c0 0x7f01580b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f0157fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f01581489c0 0x7f01580b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55542/0 (socket says 172.21.15.12:55542) 2024-04-27T07:05:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f0157fff640 1 -- 172.21.15.12:0/3622752166 learned_addr learned my addr 172.21.15.12:0/3622752166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f015d08a640 1 --2- 172.21.15.12:0/3622752166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0158141900 0x7f0158141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f015c889640 1 --2- 172.21.15.12:0/3622752166 >> v2:172.21.15.53:3300/0 conn(0x7f01580b6480 0x7f01580b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f0157fff640 1 -- 172.21.15.12:0/3622752166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0158141900 msgr2=0x7f0158141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f0157fff640 1 --2- 172.21.15.12:0/3622752166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0158141900 0x7f0158141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f0157fff640 1 -- 172.21.15.12:0/3622752166 >> v2:172.21.15.53:3300/0 conn(0x7f01580b6480 msgr2=0x7f01580b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f0157fff640 1 --2- 172.21.15.12:0/3622752166 >> v2:172.21.15.53:3300/0 conn(0x7f01580b6480 0x7f01580b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.928+0000 7f0157fff640 1 -- 172.21.15.12:0/3622752166 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01581481c0 con 0x7f01581489c0 2024-04-27T07:05:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f0157fff640 1 --2- 172.21.15.12:0/3622752166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 0x7f01580b7fa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f015806c2e0 tx=0x7f014001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459adb6a5d0e05e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f01577fe640 1 -- 172.21.15.12:0/3622752166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01400b0070 con 0x7f01581489c0 2024-04-27T07:05:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f01577fe640 1 -- 172.21.15.12:0/3622752166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01400ab600 con 0x7f01581489c0 2024-04-27T07:05:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f01577fe640 1 -- 172.21.15.12:0/3622752166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01400ab9d0 con 0x7f01581489c0 2024-04-27T07:05:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 -- 172.21.15.12:0/3622752166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 msgr2=0x7f01580b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 --2- 172.21.15.12:0/3622752166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 0x7f01580b7fa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f015806c2e0 tx=0x7f014001c490 comp rx=0 tx=0).stop 2024-04-27T07:05:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 -- 172.21.15.12:0/3622752166 shutdown_connections 2024-04-27T07:05:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 --2- 172.21.15.12:0/3622752166 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f0158141900 0x7f0158141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 --2- 172.21.15.12:0/3622752166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 0x7f01580b7fa0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 --2- 172.21.15.12:0/3622752166 >> v2:172.21.15.53:3300/0 conn(0x7f01580b6480 0x7f01580b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 -- 172.21.15.12:0/3622752166 >> 172.21.15.12:0/3622752166 conn(0x7f01580bd620 msgr2=0x7f01580bda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 -- 172.21.15.12:0/3622752166 shutdown_connections 2024-04-27T07:05:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 -- 172.21.15.12:0/3622752166 wait complete. 2024-04-27T07:05:27.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 Processor -- start 2024-04-27T07:05:27.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 -- start start 2024-04-27T07:05:27.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01580b6480 0x7f015813b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0158141900 0x7f0158144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 0x7f0158144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f01580b51c0 con 0x7f01581489c0 2024-04-27T07:05:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01580b5040 con 0x7f0158141900 2024-04-27T07:05:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015eaed640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f015814bad0 con 0x7f01580b6480 2024-04-27T07:05:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015d08a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 0x7f0158144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015d08a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 0x7f0158144e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55556/0 (socket says 172.21.15.12:55556) 2024-04-27T07:05:27.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015d08a640 1 -- 172.21.15.12:0/2816480554 learned_addr learned my addr 172.21.15.12:0/2816480554 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f0157fff640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0158141900 0x7f0158144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.932+0000 7f015c889640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01580b6480 0x7f015813b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015d08a640 1 -- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01580b6480 msgr2=0x7f015813b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015d08a640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01580b6480 0x7f015813b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015d08a640 1 -- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0158141900 msgr2=0x7f0158144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015d08a640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0158141900 0x7f0158144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015d08a640 1 -- 172.21.15.12:0/2816480554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0158143640 con 0x7f01581489c0 2024-04-27T07:05:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015d08a640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 0x7f0158144e50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0150007e40 tx=0x7f015000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:27.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f01557fa640 1 -- 172.21.15.12:0/2816480554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150014620 con 0x7f01581489c0 2024-04-27T07:05:27.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f01557fa640 1 -- 172.21.15.12:0/2816480554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0150019070 con 0x7f01581489c0 2024-04-27T07:05:27.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015eaed640 1 -- 172.21.15.12:0/2816480554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0158143920 con 0x7f01581489c0 2024-04-27T07:05:27.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f01557fa640 1 -- 172.21.15.12:0/2816480554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150014a20 con 0x7f01581489c0 2024-04-27T07:05:27.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015eaed640 1 -- 172.21.15.12:0/2816480554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015814aec0 con 0x7f01581489c0 2024-04-27T07:05:27.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015eaed640 1 -- 172.21.15.12:0/2816480554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0158068c20 con 0x7f01581489c0 2024-04-27T07:05:27.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f01557fa640 1 -- 172.21.15.12:0/2816480554 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0150014bc0 con 0x7f01581489c0 2024-04-27T07:05:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f01557fa640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0138068380 0x7f013806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f01557fa640 1 -- 172.21.15.12:0/2816480554 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f0150086ea0 con 0x7f01581489c0 2024-04-27T07:05:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015c889640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0138068380 0x7f013806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:27.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.936+0000 7f015c889640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0138068380 0x7f013806a840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f014800b9d0 tx=0x7f0148008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:27.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:27.940+0000 7f01557fa640 1 -- 172.21.15.12:0/2816480554 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f01500577f0 con 0x7f01581489c0 2024-04-27T07:05:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.096+0000 7f015eaed640 1 -- 172.21.15.12:0/2816480554 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f0158140cf0 con 0x7f0138068380 2024-04-27T07:05:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.096+0000 7f01557fa640 1 -- 172.21.15.12:0/2816480554 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0158140cf0 con 0x7f0138068380 2024-04-27T07:05:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 0 to deep-scrub 2024-04-27T07:05:28.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 -- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0138068380 msgr2=0x7f013806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0138068380 0x7f013806a840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f014800b9d0 tx=0x7f0148008040 comp rx=0 tx=0).stop 2024-04-27T07:05:28.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 -- 172.21.15.12:0/2816480554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 msgr2=0x7f0158144e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 0x7f0158144e50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0150007e40 tx=0x7f015000e4b0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 -- 172.21.15.12:0/2816480554 shutdown_connections 2024-04-27T07:05:28.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f0138068380 0x7f013806a840 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01581489c0 0x7f0158144e50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0158141900 0x7f0158144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 --2- 172.21.15.12:0/2816480554 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f01580b6480 0x7f015813b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 -- 172.21.15.12:0/2816480554 >> 172.21.15.12:0/2816480554 conn(0x7f01580bd620 msgr2=0x7f015813f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:28.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 -- 172.21.15.12:0/2816480554 shutdown_connections 2024-04-27T07:05:28.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.100+0000 7f015eaed640 1 -- 172.21.15.12:0/2816480554 wait complete. 2024-04-27T07:05:28.123 INFO:tasks.ceph:Scrubbing osd.1 2024-04-27T07:05:28.123 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:05:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5afabb0640 1 Processor -- start 2024-04-27T07:05:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5afabb0640 1 -- start start 2024-04-27T07:05:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5afabb0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f5af40b51b0 0x7f5af40af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5afabb0640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af409ce70 con 0x7f5af409d570 2024-04-27T07:05:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5afabb0640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af40b5590 con 0x7f5af409d9e0 2024-04-27T07:05:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5afabb0640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f5af409e770 con 0x7f5af40b51b0 2024-04-27T07:05:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af3fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5af409d570 0x7f5af4144fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52226/0 (socket says 172.21.15.12:52226) 2024-04-27T07:05:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af3fff640 1 -- 172.21.15.12:0/1925608428 learned_addr learned my addr 172.21.15.12:0/1925608428 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af894c640 1 --2- 172.21.15.12:0/1925608428 >> v2:172.21.15.53:3300/0 conn(0x7f5af40b51b0 0x7f5af40af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 373670274 0 0) 0x7f5af409ce70 con 0x7f5af409d570 2024-04-27T07:05:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ae0003530 con 0x7f5af409d570 2024-04-27T07:05:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3392329725 0 0) 0x7f5ae0003530 con 0x7f5af409d570 2024-04-27T07:05:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 >> v2:172.21.15.53:3300/0 conn(0x7f5af40b51b0 msgr2=0x7f5af40af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 --2- 172.21.15.12:0/1925608428 >> v2:172.21.15.53:3300/0 conn(0x7f5af40b51b0 0x7f5af40af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 >> v1:172.21.15.53:6789/0 conn(0x7f5af409d9e0 legacy=0x7f5af414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af40afbb0 con 0x7f5af409d570 2024-04-27T07:05:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f5adc001680 con 0x7f5af409d570 2024-04-27T07:05:28.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 >> v1:172.21.15.12:6789/0 conn(0x7f5af409d570 legacy=0x7f5af4144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae00039c0 0x7f5ae0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5ae0004810 0x7f5ae0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 0x7f5ae000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5af409e770 con 0x7f5ae00051d0 2024-04-27T07:05:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae000c070 con 0x7f5ae00039c0 2024-04-27T07:05:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae000c270 con 0x7f5ae0004810 2024-04-27T07:05:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af914d640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 0x7f5ae000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af3fff640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5ae0004810 0x7f5ae0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af894c640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae00039c0 0x7f5ae0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af914d640 1 -- 172.21.15.12:0/1925608428 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5ae0004810 msgr2=0x7f5ae0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:28.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af914d640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5ae0004810 0x7f5ae0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af914d640 1 -- 172.21.15.12:0/1925608428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae00039c0 msgr2=0x7f5ae0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:28.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af914d640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5ae00039c0 0x7f5ae0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.244+0000 7f5af914d640 1 -- 172.21.15.12:0/1925608428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae0003530 con 0x7f5ae00051d0 2024-04-27T07:05:28.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5af914d640 1 -- 172.21.15.12:0/1925608428 reap_dead start 2024-04-27T07:05:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5af914d640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 0x7f5ae000b9d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4001780 tx=0x7f5ae401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df92990ca4b55a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40013a0 con 0x7f5ae00051d0 2024-04-27T07:05:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae40a9070 con 0x7f5ae00051d0 2024-04-27T07:05:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5af37fe640 1 -- 172.21.15.12:0/1925608428 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4003600 con 0x7f5ae00051d0 2024-04-27T07:05:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5afabb0640 1 -- 172.21.15.12:0/1925608428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 msgr2=0x7f5ae000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5afabb0640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 0x7f5ae000b9d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4001780 tx=0x7f5ae401c420 comp rx=0 tx=0).stop 2024-04-27T07:05:28.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5afabb0640 1 -- 172.21.15.12:0/1925608428 shutdown_connections 2024-04-27T07:05:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5afabb0640 1 --2- 172.21.15.12:0/1925608428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 0x7f5ae000b9d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5afabb0640 1 -- 172.21.15.12:0/1925608428 >> 172.21.15.12:0/1925608428 conn(0x7f5af40b9840 msgr2=0x7f5af40b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5afabb0640 1 -- 172.21.15.12:0/1925608428 shutdown_connections 2024-04-27T07:05:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5afabb0640 1 -- 172.21.15.12:0/1925608428 wait complete. 2024-04-27T07:05:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.248+0000 7f5afabb0640 1 Processor -- start 2024-04-27T07:05:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5afabb0640 1 -- start start 2024-04-27T07:05:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5afabb0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 0x7f5af41ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5afabb0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af41dcd80 0x7f5af41cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5afabb0640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5af409b4c0 0x7f5af409b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5afabb0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5af40b00d0 con 0x7f5ae00051d0 2024-04-27T07:05:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5afabb0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5af409c4f0 con 0x7f5af41dcd80 2024-04-27T07:05:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5afabb0640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f5af409be80 con 0x7f5af409b4c0 2024-04-27T07:05:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af894c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 0x7f5af41ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af894c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 0x7f5af41ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55574/0 (socket says 172.21.15.12:55574) 2024-04-27T07:05:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af894c640 1 -- 172.21.15.12:0/2896424799 learned_addr learned my addr 172.21.15.12:0/2896424799 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:28.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af914d640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5af409b4c0 0x7f5af409b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af3fff640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af41dcd80 0x7f5af41cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af894c640 1 -- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5af409b4c0 msgr2=0x7f5af409b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af894c640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f5af409b4c0 0x7f5af409b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af894c640 1 -- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af41dcd80 msgr2=0x7f5af41cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af894c640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5af41dcd80 0x7f5af41cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af894c640 1 -- 172.21.15.12:0/2896424799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af409c310 con 0x7f5ae00051d0 2024-04-27T07:05:28.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af894c640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 0x7f5af41ceb50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009fb0 tx=0x7f5aec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af17fa640 1 -- 172.21.15.12:0/2896424799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec01a020 con 0x7f5ae00051d0 2024-04-27T07:05:28.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af17fa640 1 -- 172.21.15.12:0/2896424799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aec016470 con 0x7f5ae00051d0 2024-04-27T07:05:28.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af40b3760 con 0x7f5ae00051d0 2024-04-27T07:05:28.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af17fa640 1 -- 172.21.15.12:0/2896424799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec016860 con 0x7f5ae00051d0 2024-04-27T07:05:28.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af41ce3e0 con 0x7f5ae00051d0 2024-04-27T07:05:28.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5af41cd230 con 0x7f5ae00051d0 2024-04-27T07:05:28.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af17fa640 1 -- 172.21.15.12:0/2896424799 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5aec01d020 con 0x7f5ae00051d0 2024-04-27T07:05:28.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af17fa640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5ad40683b0 0x7f5ad406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af17fa640 1 -- 172.21.15.12:0/2896424799 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f5aec087e90 con 0x7f5ae00051d0 2024-04-27T07:05:28.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af17fa640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] conn(0x7f5ad406e270 0x7f5ad4070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af17fa640 1 -- 172.21.15.12:0/2896424799 --> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5aec009200 con 0x7f5ad406e270 2024-04-27T07:05:28.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.252+0000 7f5af17fa640 1 -- 172.21.15.12:0/2896424799 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5aec088320 con 0x7f5ae00051d0 2024-04-27T07:05:28.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.256+0000 7f5af3fff640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5ad40683b0 0x7f5ad406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.256+0000 7f5af914d640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] conn(0x7f5ad406e270 0x7f5ad4070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.256+0000 7f5af914d640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] conn(0x7f5ad406e270 0x7f5ad4070690 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.256+0000 7f5af3fff640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5ad40683b0 0x7f5ad406a870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5adc001060 tx=0x7f5adc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.256+0000 7f5af17fa640 1 -- 172.21.15.12:0/2896424799 <== osd.1 v2:172.21.15.12:6800/3260695366 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5aec009200 con 0x7f5ad406e270 2024-04-27T07:05:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 --> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5af4068d00 con 0x7f5ad406e270 2024-04-27T07:05:28.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5af17fa640 1 -- 172.21.15.12:0/2896424799 <== osd.1 v2:172.21.15.12:6800/3260695366 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5af4068d00 con 0x7f5ad406e270 2024-04-27T07:05:28.275 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:05:28.275 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:05:28.275 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:05:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 >> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] conn(0x7f5ad406e270 msgr2=0x7f5ad4070690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.12:6800/3260695366,v1:172.21.15.12:6801/3260695366] conn(0x7f5ad406e270 0x7f5ad4070690 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5ad40683b0 msgr2=0x7f5ad406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f5ad40683b0 0x7f5ad406a870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5adc001060 tx=0x7f5adc009040 comp rx=0 tx=0).stop 2024-04-27T07:05:28.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 msgr2=0x7f5af41ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 --2- 172.21.15.12:0/2896424799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae00051d0 0x7f5af41ceb50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009fb0 tx=0x7f5aec0115a0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5af914d640 1 -- 172.21.15.12:0/2896424799 reap_dead start 2024-04-27T07:05:28.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 shutdown_connections 2024-04-27T07:05:28.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 >> 172.21.15.12:0/2896424799 conn(0x7f5af40b9840 msgr2=0x7f5af40b9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:28.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 shutdown_connections 2024-04-27T07:05:28.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.268+0000 7f5afabb0640 1 -- 172.21.15.12:0/2896424799 wait complete. 2024-04-27T07:05:28.290 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-27T07:05:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe512dd2640 1 Processor -- start 2024-04-27T07:05:28.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe512dd2640 1 -- start start 2024-04-27T07:05:28.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe512dd2640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe50c06e370 0x7fe50c13faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe512dd2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe50c0b3650 0x7fe50c0b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe512dd2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe50c0b4d70 con 0x7fe50c06d300 2024-04-27T07:05:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe512dd2640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fe50c0740d0 con 0x7fe50c0b3650 2024-04-27T07:05:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe512dd2640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe50c0b2bf0 con 0x7fe50c06e370 2024-04-27T07:05:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe50bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe50c0b3650 0x7fe50c0b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe50bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe50c0b3650 0x7fe50c0b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55594/0 (socket says 172.21.15.12:55594) 2024-04-27T07:05:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe50bfff640 1 -- 172.21.15.12:0/594299553 learned_addr learned my addr 172.21.15.12:0/594299553 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe510b6e640 1 --2- 172.21.15.12:0/594299553 >> v2:172.21.15.53:3300/0 conn(0x7fe50c06e370 0x7fe50c13faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe50bfff640 1 -- 172.21.15.12:0/594299553 >> v2:172.21.15.53:3300/0 conn(0x7fe50c06e370 msgr2=0x7fe50c13faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe50bfff640 1 --2- 172.21.15.12:0/594299553 >> v2:172.21.15.53:3300/0 conn(0x7fe50c06e370 0x7fe50c13faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe50bfff640 1 -- 172.21.15.12:0/594299553 >> v1:172.21.15.53:6789/0 conn(0x7fe50c06d300 legacy=0x7fe50c0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.400+0000 7fe50bfff640 1 -- 172.21.15.12:0/594299553 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c073e60 con 0x7fe50c0b3650 2024-04-27T07:05:28.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe50bfff640 1 --2- 172.21.15.12:0/594299553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0b3650 0x7fe50c0b5190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe50c069d80 tx=0x7fe4fc01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1864f986ba3fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe50b7fe640 1 -- 172.21.15.12:0/594299553 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0a5690 con 0x7fe50c0b3650 2024-04-27T07:05:28.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe50b7fe640 1 -- 172.21.15.12:0/594299553 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4fc0a5830 con 0x7fe50c0b3650 2024-04-27T07:05:28.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe50b7fe640 1 -- 172.21.15.12:0/594299553 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0a5c00 con 0x7fe50c0b3650 2024-04-27T07:05:28.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe512dd2640 1 -- 172.21.15.12:0/594299553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0b3650 msgr2=0x7fe50c0b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe512dd2640 1 --2- 172.21.15.12:0/594299553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0b3650 0x7fe50c0b5190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe50c069d80 tx=0x7fe4fc01c920 comp rx=0 tx=0).stop 2024-04-27T07:05:28.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe512dd2640 1 -- 172.21.15.12:0/594299553 shutdown_connections 2024-04-27T07:05:28.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe512dd2640 1 --2- 172.21.15.12:0/594299553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c0b3650 0x7fe50c0b5190 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe512dd2640 1 --2- 172.21.15.12:0/594299553 >> v2:172.21.15.53:3300/0 conn(0x7fe50c06e370 0x7fe50c13faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe512dd2640 1 -- 172.21.15.12:0/594299553 >> 172.21.15.12:0/594299553 conn(0x7fe50c0b93b0 msgr2=0x7fe50c0b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:28.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe512dd2640 1 -- 172.21.15.12:0/594299553 shutdown_connections 2024-04-27T07:05:28.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe512dd2640 1 -- 172.21.15.12:0/594299553 wait complete. 2024-04-27T07:05:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe512dd2640 1 Processor -- start 2024-04-27T07:05:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.404+0000 7fe512dd2640 1 -- start start 2024-04-27T07:05:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe512dd2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe50c06d300 0x7fe50c0b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe512dd2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c06e370 0x7fe50c138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe512dd2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe50c0b3650 0x7fe50c09c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe512dd2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1535c0 con 0x7fe50c06e370 2024-04-27T07:05:28.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe512dd2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c153440 con 0x7fe50c06d300 2024-04-27T07:05:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe512dd2640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe50c153740 con 0x7fe50c0b3650 2024-04-27T07:05:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c06e370 0x7fe50c138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c06e370 0x7fe50c138930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55596/0 (socket says 172.21.15.12:55596) 2024-04-27T07:05:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe50bfff640 1 -- 172.21.15.12:0/1532698989 learned_addr learned my addr 172.21.15.12:0/1532698989 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe51136f640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe50c0b3650 0x7fe50c09c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe510b6e640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe50c06d300 0x7fe50c0b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe50bfff640 1 -- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe50c0b3650 msgr2=0x7fe50c09c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe50bfff640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe50c0b3650 0x7fe50c09c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe50bfff640 1 -- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe50c06d300 msgr2=0x7fe50c0b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe50bfff640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe50c06d300 0x7fe50c0b7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe50bfff640 1 -- 172.21.15.12:0/1532698989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c0b8fd0 con 0x7fe50c06e370 2024-04-27T07:05:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe51136f640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe50c0b3650 0x7fe50c09c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe50bfff640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c06e370 0x7fe50c138930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc01afd0 tx=0x7fe4fc0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe5097fa640 1 -- 172.21.15.12:0/1532698989 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0bc020 con 0x7fe50c06e370 2024-04-27T07:05:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe5097fa640 1 -- 172.21.15.12:0/1532698989 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4fc0b8670 con 0x7fe50c06e370 2024-04-27T07:05:28.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe512dd2640 1 -- 172.21.15.12:0/1532698989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c139590 con 0x7fe50c06e370 2024-04-27T07:05:28.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe5097fa640 1 -- 172.21.15.12:0/1532698989 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0b8a40 con 0x7fe50c06e370 2024-04-27T07:05:28.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe512dd2640 1 -- 172.21.15.12:0/1532698989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c0b8c90 con 0x7fe50c06e370 2024-04-27T07:05:28.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe512dd2640 1 -- 172.21.15.12:0/1532698989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50c0b7790 con 0x7fe50c06e370 2024-04-27T07:05:28.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe5097fa640 1 -- 172.21.15.12:0/1532698989 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe4fc0bf020 con 0x7fe50c06e370 2024-04-27T07:05:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe5097fa640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe4ec068380 0x7fe4ec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe5097fa640 1 -- 172.21.15.12:0/1532698989 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fe4fc121af0 con 0x7fe50c06e370 2024-04-27T07:05:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.408+0000 7fe510b6e640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe4ec068380 0x7fe4ec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.412+0000 7fe510b6e640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe4ec068380 0x7fe4ec06a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400b9d0 tx=0x7fe4f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.412+0000 7fe5097fa640 1 -- 172.21.15.12:0/1532698989 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe4fc0c6030 con 0x7fe50c06e370 2024-04-27T07:05:28.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.568+0000 7fe512dd2640 1 -- 172.21.15.12:0/1532698989 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fe50c068c20 con 0x7fe4ec068380 2024-04-27T07:05:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.568+0000 7fe5097fa640 1 -- 172.21.15.12:0/1532698989 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe50c068c20 con 0x7fe4ec068380 2024-04-27T07:05:28.576 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 1 to deep-scrub 2024-04-27T07:05:28.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 -- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe4ec068380 msgr2=0x7fe4ec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe4ec068380 0x7fe4ec06a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400b9d0 tx=0x7fe4f4008040 comp rx=0 tx=0).stop 2024-04-27T07:05:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 -- 172.21.15.12:0/1532698989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c06e370 msgr2=0x7fe50c138930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c06e370 0x7fe50c138930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc01afd0 tx=0x7fe4fc0196c0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 -- 172.21.15.12:0/1532698989 shutdown_connections 2024-04-27T07:05:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe4ec068380 0x7fe4ec06a840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe50c0b3650 0x7fe50c09c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe50c06e370 0x7fe50c138930 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 --2- 172.21.15.12:0/1532698989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe50c06d300 0x7fe50c0b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 -- 172.21.15.12:0/1532698989 >> 172.21.15.12:0/1532698989 conn(0x7fe50c0b93b0 msgr2=0x7fe50c139a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:28.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 -- 172.21.15.12:0/1532698989 shutdown_connections 2024-04-27T07:05:28.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.572+0000 7fe512dd2640 1 -- 172.21.15.12:0/1532698989 wait complete. 2024-04-27T07:05:28.594 INFO:tasks.ceph:Scrubbing osd.2 2024-04-27T07:05:28.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:05:28.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f35707c8640 1 Processor -- start 2024-04-27T07:05:28.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f35707c8640 1 -- start start 2024-04-27T07:05:28.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f35707c8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f356806d760 0x7f35680ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f35707c8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f35681505f0 0x7f35681571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f35707c8640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f356809da20 con 0x7f356806db40 2024-04-27T07:05:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f35707c8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f35680b3e40 con 0x7f356806d760 2024-04-27T07:05:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f35707c8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f356809d590 con 0x7f35681505f0 2024-04-27T07:05:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f356e564640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f356806d760 0x7f35680ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f356e564640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f356806d760 0x7f35680ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55602/0 (socket says 172.21.15.12:55602) 2024-04-27T07:05:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f356e564640 1 -- 172.21.15.12:0/1468001436 learned_addr learned my addr 172.21.15.12:0/1468001436 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f356ed65640 1 --2- 172.21.15.12:0/1468001436 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f35681505f0 0x7f35681571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f356e564640 1 -- 172.21.15.12:0/1468001436 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f35681505f0 msgr2=0x7f35681571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:28.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f356e564640 1 --2- 172.21.15.12:0/1468001436 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f35681505f0 0x7f35681571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f356e564640 1 -- 172.21.15.12:0/1468001436 >> v1:172.21.15.53:6789/0 conn(0x7f356806db40 legacy=0x7f356814fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:28.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.712+0000 7f356e564640 1 -- 172.21.15.12:0/1468001436 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35680b0240 con 0x7f356806d760 2024-04-27T07:05:28.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356e564640 1 --2- 172.21.15.12:0/1468001436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f356806d760 0x7f35680ae680 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3564009870 tx=0x7f356401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f507e2abdd495732 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356d562640 1 -- 172.21.15.12:0/1468001436 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35640b0070 con 0x7f356806d760 2024-04-27T07:05:28.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356d562640 1 -- 172.21.15.12:0/1468001436 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35640ab600 con 0x7f356806d760 2024-04-27T07:05:28.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356d562640 1 -- 172.21.15.12:0/1468001436 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35640ab9d0 con 0x7f356806d760 2024-04-27T07:05:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- 172.21.15.12:0/1468001436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f356806d760 msgr2=0x7f35680ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 --2- 172.21.15.12:0/1468001436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f356806d760 0x7f35680ae680 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3564009870 tx=0x7f356401c490 comp rx=0 tx=0).stop 2024-04-27T07:05:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- 172.21.15.12:0/1468001436 shutdown_connections 2024-04-27T07:05:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 --2- 172.21.15.12:0/1468001436 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f35681505f0 0x7f35681571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 --2- 172.21.15.12:0/1468001436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f356806d760 0x7f35680ae680 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- 172.21.15.12:0/1468001436 >> 172.21.15.12:0/1468001436 conn(0x7f35680b9840 msgr2=0x7f35680b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- 172.21.15.12:0/1468001436 shutdown_connections 2024-04-27T07:05:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- 172.21.15.12:0/1468001436 wait complete. 2024-04-27T07:05:28.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 Processor -- start 2024-04-27T07:05:28.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- start start 2024-04-27T07:05:28.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f356806d760 0x7f356808fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f356806db40 0x7f356808dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f35681505f0 0x7f356814b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3568074200 con 0x7f356806d760 2024-04-27T07:05:28.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3568074080 con 0x7f356806db40 2024-04-27T07:05:28.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f35680b5aa0 con 0x7f35681505f0 2024-04-27T07:05:28.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356e564640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f356806d760 0x7f356808fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356e564640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f356806d760 0x7f356808fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55618/0 (socket says 172.21.15.12:55618) 2024-04-27T07:05:28.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356e564640 1 -- 172.21.15.12:0/3216757673 learned_addr learned my addr 172.21.15.12:0/3216757673 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:28.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356e564640 1 -- 172.21.15.12:0/3216757673 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f35681505f0 msgr2=0x7f356814b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356e564640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f35681505f0 0x7f356814b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356e564640 1 -- 172.21.15.12:0/3216757673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f356806db40 msgr2=0x7f356808dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356e564640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f356806db40 0x7f356808dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356e564640 1 -- 172.21.15.12:0/3216757673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356808d0b0 con 0x7f356806d760 2024-04-27T07:05:28.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f356e564640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f356806d760 0x7f356808fc20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f35640ae040 tx=0x7f356401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f355effd640 1 -- 172.21.15.12:0/3216757673 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35640ba020 con 0x7f356806d760 2024-04-27T07:05:28.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f355effd640 1 -- 172.21.15.12:0/3216757673 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35640b6640 con 0x7f356806d760 2024-04-27T07:05:28.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f355effd640 1 -- 172.21.15.12:0/3216757673 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35640b0040 con 0x7f356806d760 2024-04-27T07:05:28.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356808f1c0 con 0x7f356806d760 2024-04-27T07:05:28.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.716+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356808d350 con 0x7f356806d760 2024-04-27T07:05:28.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f356808cb60 con 0x7f356806d760 2024-04-27T07:05:28.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f355effd640 1 -- 172.21.15.12:0/3216757673 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f35640bd020 con 0x7f356806d760 2024-04-27T07:05:28.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f355effd640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f354c068380 0x7f354c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f355effd640 1 -- 172.21.15.12:0/3216757673 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f3564120d30 con 0x7f356806d760 2024-04-27T07:05:28.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f355effd640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] conn(0x7f354c06e240 0x7f354c070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f355effd640 1 -- 172.21.15.12:0/3216757673 --> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35640c6e50 con 0x7f354c06e240 2024-04-27T07:05:28.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f355effd640 1 -- 172.21.15.12:0/3216757673 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3564121190 con 0x7f356806d760 2024-04-27T07:05:28.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f356ed65640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] conn(0x7f354c06e240 0x7f354c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f356dd63640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f354c068380 0x7f354c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f356ed65640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] conn(0x7f354c06e240 0x7f354c070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f356dd63640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f354c068380 0x7f354c06a840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f356809cf20 tx=0x7f35580079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.720+0000 7f355effd640 1 -- 172.21.15.12:0/3216757673 <== osd.2 v2:172.21.15.12:6811/3457925485 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35640c6e50 con 0x7f354c06e240 2024-04-27T07:05:28.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.732+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 --> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3568068d00 con 0x7f354c06e240 2024-04-27T07:05:28.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f355effd640 1 -- 172.21.15.12:0/3216757673 <== osd.2 v2:172.21.15.12:6811/3457925485 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3568068d00 con 0x7f354c06e240 2024-04-27T07:05:28.741 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:05:28.741 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:05:28.741 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:05:28.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 >> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] conn(0x7f354c06e240 msgr2=0x7f354c070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f35707c8640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.12:6811/3457925485,v1:172.21.15.12:6815/3457925485] conn(0x7f354c06e240 0x7f354c070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f354c068380 msgr2=0x7f354c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f35707c8640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f354c068380 0x7f354c06a840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f356809cf20 tx=0x7f35580079c0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f356806d760 msgr2=0x7f356808fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f35707c8640 1 --2- 172.21.15.12:0/3216757673 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f356806d760 0x7f356808fc20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f35640ae040 tx=0x7f356401acd0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f356ed65640 1 -- 172.21.15.12:0/3216757673 reap_dead start 2024-04-27T07:05:28.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 shutdown_connections 2024-04-27T07:05:28.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 >> 172.21.15.12:0/3216757673 conn(0x7f35680b9840 msgr2=0x7f35680b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:28.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 shutdown_connections 2024-04-27T07:05:28.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.736+0000 7f35707c8640 1 -- 172.21.15.12:0/3216757673 wait complete. 2024-04-27T07:05:28.756 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-27T07:05:28.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.860+0000 7f364dc17640 1 Processor -- start 2024-04-27T07:05:28.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.860+0000 7f364dc17640 1 -- start start 2024-04-27T07:05:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f364dc17640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f364806e410 0x7f3648150350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f364dc17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3648144cb0 con 0x7f36480b4ae0 2024-04-27T07:05:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f364dc17640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3648144f90 con 0x7f36480b3db0 2024-04-27T07:05:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f364dc17640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3648144a80 con 0x7f364806e410 2024-04-27T07:05:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36477fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f36480b4ae0 0x7f36481454b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52230/0 (socket says 172.21.15.12:52230) 2024-04-27T07:05:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36477fe640 1 -- 172.21.15.12:0/844014866 learned_addr learned my addr 172.21.15.12:0/844014866 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f3647fff640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f364806e410 0x7f3648150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3159935379 0 0) 0x7f3648144cb0 con 0x7f36480b4ae0 2024-04-27T07:05:28.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3634003530 con 0x7f36480b4ae0 2024-04-27T07:05:28.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2729875489 0 0) 0x7f3634003530 con 0x7f36480b4ae0 2024-04-27T07:05:28.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f364806e410 msgr2=0x7f3648150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:28.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f364806e410 0x7f3648150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 >> v1:172.21.15.53:6789/0 conn(0x7f36480b3db0 legacy=0x7f364809da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3648073de0 con 0x7f36480b4ae0 2024-04-27T07:05:28.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f3630001680 con 0x7f36480b4ae0 2024-04-27T07:05:28.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 >> v1:172.21.15.12:6789/0 conn(0x7f36480b4ae0 legacy=0x7f36481454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f36340039f0 0x7f3634003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3634004810 0x7f3634004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36340051d0 0x7f363400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f363400c0a0 con 0x7f36340051d0 2024-04-27T07:05:28.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f363400c220 con 0x7f3634004810 2024-04-27T07:05:28.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f363400e630 con 0x7f36340039f0 2024-04-27T07:05:28.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f3647fff640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f364806e410 0x7f3648150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:28.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f3647fff640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36340051d0 0x7f363400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f3646ffd640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3634004810 0x7f3634004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f3647fff640 1 -- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f36340039f0 msgr2=0x7f3634003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:28.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f3647fff640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f36340039f0 0x7f3634003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f3647fff640 1 -- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3634004810 msgr2=0x7f3634004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:28.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f3647fff640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3634004810 0x7f3634004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.864+0000 7f3647fff640 1 -- 172.21.15.12:0/844014866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3634003530 con 0x7f36340051d0 2024-04-27T07:05:28.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3647fff640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36340051d0 0x7f363400b9d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f364806e7f0 tx=0x7f364001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562ff67ef5019cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364000f040 con 0x7f36340051d0 2024-04-27T07:05:28.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36400b0070 con 0x7f36340051d0 2024-04-27T07:05:28.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f36467fc640 1 -- 172.21.15.12:0/844014866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36400ab840 con 0x7f36340051d0 2024-04-27T07:05:28.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- 172.21.15.12:0/844014866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36340051d0 msgr2=0x7f363400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 --2- 172.21.15.12:0/844014866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36340051d0 0x7f363400b9d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f364806e7f0 tx=0x7f364001c3d0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3647fff640 1 -- 172.21.15.12:0/844014866 reap_dead start 2024-04-27T07:05:28.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- 172.21.15.12:0/844014866 shutdown_connections 2024-04-27T07:05:28.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- 172.21.15.12:0/844014866 >> 172.21.15.12:0/844014866 conn(0x7f36480b93b0 msgr2=0x7f36480b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:28.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- 172.21.15.12:0/844014866 shutdown_connections 2024-04-27T07:05:28.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- 172.21.15.12:0/844014866 wait complete. 2024-04-27T07:05:28.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 Processor -- start 2024-04-27T07:05:28.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- start start 2024-04-27T07:05:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f36340051d0 0x7f36480b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36481428f0 0x7f36481d9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3648142d50 0x7f36481d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f36480b4ec0 con 0x7f36481428f0 2024-04-27T07:05:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f364809e3a0 con 0x7f3648142d50 2024-04-27T07:05:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f364806d300 con 0x7f36340051d0 2024-04-27T07:05:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3646ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36481428f0 0x7f36481d9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3646ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36481428f0 0x7f36481d9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55630/0 (socket says 172.21.15.12:55630) 2024-04-27T07:05:28.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3646ffd640 1 -- 172.21.15.12:0/2540158089 learned_addr learned my addr 172.21.15.12:0/2540158089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:28.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3647fff640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3648142d50 0x7f36481d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f36477fe640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f36340051d0 0x7f36480b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3646ffd640 1 -- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f36340051d0 msgr2=0x7f36480b8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3646ffd640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f36340051d0 0x7f36480b8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3646ffd640 1 -- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3648142d50 msgr2=0x7f36481d99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:28.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3646ffd640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3648142d50 0x7f36481d99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:28.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3646ffd640 1 -- 172.21.15.12:0/2540158089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36480b7c30 con 0x7f36481428f0 2024-04-27T07:05:28.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f3646ffd640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36481428f0 0x7f36481d9470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f363800d910 tx=0x7f363800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f363ffff640 1 -- 172.21.15.12:0/2540158089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638003db0 con 0x7f36481428f0 2024-04-27T07:05:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f363ffff640 1 -- 172.21.15.12:0/2540158089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3638017070 con 0x7f36481428f0 2024-04-27T07:05:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- 172.21.15.12:0/2540158089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36480b7f10 con 0x7f36481428f0 2024-04-27T07:05:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f363ffff640 1 -- 172.21.15.12:0/2540158089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363800b8d0 con 0x7f36481428f0 2024-04-27T07:05:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.868+0000 7f364dc17640 1 -- 172.21.15.12:0/2540158089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36481dc690 con 0x7f36481428f0 2024-04-27T07:05:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.872+0000 7f364dc17640 1 -- 172.21.15.12:0/2540158089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3648068c20 con 0x7f36481428f0 2024-04-27T07:05:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.872+0000 7f363ffff640 1 -- 172.21.15.12:0/2540158089 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f363800ba70 con 0x7f36481428f0 2024-04-27T07:05:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.872+0000 7f363ffff640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3628068380 0x7f362806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:28.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.872+0000 7f363ffff640 1 -- 172.21.15.12:0/2540158089 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f3638086720 con 0x7f36481428f0 2024-04-27T07:05:28.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.872+0000 7f36477fe640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3628068380 0x7f362806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:28.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.872+0000 7f36477fe640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3628068380 0x7f362806a840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3630000fc0 tx=0x7f3630001650 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:28.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:28.876+0000 7f363ffff640 1 -- 172.21.15.12:0/2540158089 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3638057070 con 0x7f36481428f0 2024-04-27T07:05:29.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.028+0000 7f364dc17640 1 -- 172.21.15.12:0/2540158089 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f36480b8280 con 0x7f3628068380 2024-04-27T07:05:29.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f363ffff640 1 -- 172.21.15.12:0/2540158089 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f36480b8280 con 0x7f3628068380 2024-04-27T07:05:29.037 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 2 to deep-scrub 2024-04-27T07:05:29.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 -- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3628068380 msgr2=0x7f362806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3628068380 0x7f362806a840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3630000fc0 tx=0x7f3630001650 comp rx=0 tx=0).stop 2024-04-27T07:05:29.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 -- 172.21.15.12:0/2540158089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36481428f0 msgr2=0x7f36481d9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36481428f0 0x7f36481d9470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f363800d910 tx=0x7f363800ddd0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 -- 172.21.15.12:0/2540158089 shutdown_connections 2024-04-27T07:05:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3628068380 0x7f362806a840 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3648142d50 0x7f36481d99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36481428f0 0x7f36481d9470 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 --2- 172.21.15.12:0/2540158089 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f36340051d0 0x7f36480b8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 -- 172.21.15.12:0/2540158089 >> 172.21.15.12:0/2540158089 conn(0x7f36480b93b0 msgr2=0x7f36480ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 -- 172.21.15.12:0/2540158089 shutdown_connections 2024-04-27T07:05:29.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.032+0000 7f364dc17640 1 -- 172.21.15.12:0/2540158089 wait complete. 2024-04-27T07:05:29.055 INFO:tasks.ceph:Scrubbing osd.3 2024-04-27T07:05:29.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:05:29.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff3a789640 1 Processor -- start 2024-04-27T07:05:29.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff3a789640 1 -- start start 2024-04-27T07:05:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff3a789640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7eff3406d8e0 0x7eff3414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff3a789640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7eff341501b0 0x7eff341525a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff3a789640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff340afcb0 con 0x7eff3406d500 2024-04-27T07:05:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff3a789640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7eff340ae740 con 0x7eff341501b0 2024-04-27T07:05:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff3a789640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7eff3409e9b0 con 0x7eff3406d8e0 2024-04-27T07:05:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff38d26640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7eff341501b0 0x7eff341525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff38d26640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7eff341501b0 0x7eff341525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55640/0 (socket says 172.21.15.12:55640) 2024-04-27T07:05:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff38d26640 1 -- 172.21.15.12:0/2184436814 learned_addr learned my addr 172.21.15.12:0/2184436814 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:29.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.164+0000 7eff33fff640 1 --1- 172.21.15.12:0/2184436814 >> v1:172.21.15.12:6789/0 conn(0x7eff3406d500 0x7eff3409e2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52240/0 (socket says 172.21.15.12:52240) 2024-04-27T07:05:29.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff337fe640 1 --2- 172.21.15.12:0/2184436814 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7eff3406d8e0 0x7eff3414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff38d26640 1 -- 172.21.15.12:0/2184436814 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7eff3406d8e0 msgr2=0x7eff3414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:29.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff38d26640 1 --2- 172.21.15.12:0/2184436814 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7eff3406d8e0 0x7eff3414fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff38d26640 1 -- 172.21.15.12:0/2184436814 >> v1:172.21.15.12:6789/0 conn(0x7eff3406d500 legacy=0x7eff3409e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff32ffd640 1 -- 172.21.15.12:0/2184436814 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3753471939 0 0) 0x7eff340afcb0 con 0x7eff3406d500 2024-04-27T07:05:29.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff38d26640 1 -- 172.21.15.12:0/2184436814 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff340740d0 con 0x7eff341501b0 2024-04-27T07:05:29.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff38d26640 1 --2- 172.21.15.12:0/2184436814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff341501b0 0x7eff341525a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7eff24009870 tx=0x7eff2401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d46e2e8bc9f0ad server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff32ffd640 1 -- 172.21.15.12:0/2184436814 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff240b0070 con 0x7eff341501b0 2024-04-27T07:05:29.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff32ffd640 1 -- 172.21.15.12:0/2184436814 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff240ab600 con 0x7eff341501b0 2024-04-27T07:05:29.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff32ffd640 1 -- 172.21.15.12:0/2184436814 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff240ab9d0 con 0x7eff341501b0 2024-04-27T07:05:29.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff3a789640 1 -- 172.21.15.12:0/2184436814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff341501b0 msgr2=0x7eff341525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff3a789640 1 --2- 172.21.15.12:0/2184436814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff341501b0 0x7eff341525a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7eff24009870 tx=0x7eff2401c4d0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff3a789640 1 -- 172.21.15.12:0/2184436814 shutdown_connections 2024-04-27T07:05:29.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff3a789640 1 --2- 172.21.15.12:0/2184436814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff341501b0 0x7eff341525a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff3a789640 1 --2- 172.21.15.12:0/2184436814 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7eff3406d8e0 0x7eff3414fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff3a789640 1 -- 172.21.15.12:0/2184436814 >> 172.21.15.12:0/2184436814 conn(0x7eff340b9840 msgr2=0x7eff340b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:29.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff3a789640 1 -- 172.21.15.12:0/2184436814 shutdown_connections 2024-04-27T07:05:29.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.168+0000 7eff3a789640 1 -- 172.21.15.12:0/2184436814 wait complete. 2024-04-27T07:05:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 Processor -- start 2024-04-27T07:05:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 -- start start 2024-04-27T07:05:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff3406d500 0x7eff34072730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7eff3406d8e0 0x7eff34149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff341501b0 0x7eff34149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7eff340b5d10 con 0x7eff3406d500 2024-04-27T07:05:29.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff340b5b90 con 0x7eff341501b0 2024-04-27T07:05:29.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7eff34155300 con 0x7eff3406d8e0 2024-04-27T07:05:29.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff33fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff3406d500 0x7eff34072730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff33fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff3406d500 0x7eff34072730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55646/0 (socket says 172.21.15.12:55646) 2024-04-27T07:05:29.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff33fff640 1 -- 172.21.15.12:0/3660841106 learned_addr learned my addr 172.21.15.12:0/3660841106 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:29.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff337fe640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7eff3406d8e0 0x7eff34149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff38d26640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff341501b0 0x7eff34149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff33fff640 1 -- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7eff3406d8e0 msgr2=0x7eff34149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff33fff640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7eff3406d8e0 0x7eff34149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff33fff640 1 -- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff341501b0 msgr2=0x7eff34149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff33fff640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff341501b0 0x7eff34149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff33fff640 1 -- 172.21.15.12:0/3660841106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff341e3b60 con 0x7eff3406d500 2024-04-27T07:05:29.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff33fff640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff3406d500 0x7eff34072730 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00c7c0 tx=0x7eff2c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff30ff9640 1 -- 172.21.15.12:0/3660841106 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c018070 con 0x7eff3406d500 2024-04-27T07:05:29.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff30ff9640 1 -- 172.21.15.12:0/3660841106 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff2c00a5e0 con 0x7eff3406d500 2024-04-27T07:05:29.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff341e3e30 con 0x7eff3406d500 2024-04-27T07:05:29.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff30ff9640 1 -- 172.21.15.12:0/3660841106 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c00a9d0 con 0x7eff3406d500 2024-04-27T07:05:29.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff340706b0 con 0x7eff3406d500 2024-04-27T07:05:29.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff34068d00 con 0x7eff3406d500 2024-04-27T07:05:29.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff30ff9640 1 -- 172.21.15.12:0/3660841106 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7eff2c01b070 con 0x7eff3406d500 2024-04-27T07:05:29.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff30ff9640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7eff20068380 0x7eff2006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff30ff9640 1 -- 172.21.15.12:0/3660841106 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7eff2c085bd0 con 0x7eff3406d500 2024-04-27T07:05:29.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff30ff9640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] conn(0x7eff2006e240 0x7eff20070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff30ff9640 1 -- 172.21.15.12:0/3660841106 --> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff2c01be50 con 0x7eff2006e240 2024-04-27T07:05:29.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff30ff9640 1 -- 172.21.15.12:0/3660841106 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7eff2c08a2c0 con 0x7eff3406d500 2024-04-27T07:05:29.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff38d26640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] conn(0x7eff2006e240 0x7eff20070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.172+0000 7eff337fe640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7eff20068380 0x7eff2006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.176+0000 7eff38d26640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] conn(0x7eff2006e240 0x7eff20070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.176+0000 7eff337fe640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7eff20068380 0x7eff2006a840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff34071200 tx=0x7eff28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.176+0000 7eff30ff9640 1 -- 172.21.15.12:0/3660841106 <== osd.3 v2:172.21.15.12:6808/2279477907 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff2c01be50 con 0x7eff2006e240 2024-04-27T07:05:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 --> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7eff340b6880 con 0x7eff2006e240 2024-04-27T07:05:29.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff30ff9640 1 -- 172.21.15.12:0/3660841106 <== osd.3 v2:172.21.15.12:6808/2279477907 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7eff340b6880 con 0x7eff2006e240 2024-04-27T07:05:29.195 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:05:29.196 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:05:29.197 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:05:29.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 >> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] conn(0x7eff2006e240 msgr2=0x7eff20070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.12:6808/2279477907,v1:172.21.15.12:6809/2279477907] conn(0x7eff2006e240 0x7eff20070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7eff20068380 msgr2=0x7eff2006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7eff20068380 0x7eff2006a840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff34071200 tx=0x7eff28008040 comp rx=0 tx=0).stop 2024-04-27T07:05:29.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff3406d500 msgr2=0x7eff34072730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 --2- 172.21.15.12:0/3660841106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff3406d500 0x7eff34072730 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00c7c0 tx=0x7eff2c00cc80 comp rx=0 tx=0).stop 2024-04-27T07:05:29.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff38d26640 1 -- 172.21.15.12:0/3660841106 reap_dead start 2024-04-27T07:05:29.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 shutdown_connections 2024-04-27T07:05:29.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 >> 172.21.15.12:0/3660841106 conn(0x7eff340b9840 msgr2=0x7eff3414bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:29.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 shutdown_connections 2024-04-27T07:05:29.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.188+0000 7eff3a789640 1 -- 172.21.15.12:0/3660841106 wait complete. 2024-04-27T07:05:29.209 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-27T07:05:29.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe265091640 1 Processor -- start 2024-04-27T07:05:29.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe265091640 1 -- start start 2024-04-27T07:05:29.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe265091640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe2600b9f20 0x7fe2600b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe265091640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe2600b86f0 0x7fe2600b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe265091640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2600b7810 con 0x7fe2600b7430 2024-04-27T07:05:29.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe265091640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fe2600b9060 con 0x7fe2600b86f0 2024-04-27T07:05:29.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe265091640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe2600b5cc0 con 0x7fe2600b9f20 2024-04-27T07:05:29.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25f59e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe2600b86f0 0x7fe2600b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25f59e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe2600b86f0 0x7fe2600b8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55648/0 (socket says 172.21.15.12:55648) 2024-04-27T07:05:29.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25f59e640 1 -- 172.21.15.12:0/2356725424 learned_addr learned my addr 172.21.15.12:0/2356725424 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:29.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25e59c640 1 --2- 172.21.15.12:0/2356725424 >> v2:172.21.15.53:3300/0 conn(0x7fe2600b9f20 0x7fe2600b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25f59e640 1 -- 172.21.15.12:0/2356725424 >> v2:172.21.15.53:3300/0 conn(0x7fe2600b9f20 msgr2=0x7fe2600b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:29.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25f59e640 1 --2- 172.21.15.12:0/2356725424 >> v2:172.21.15.53:3300/0 conn(0x7fe2600b9f20 0x7fe2600b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25f59e640 1 -- 172.21.15.12:0/2356725424 >> v1:172.21.15.53:6789/0 conn(0x7fe2600b7430 legacy=0x7fe260145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:29.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25f59e640 1 -- 172.21.15.12:0/2356725424 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2600b5fa0 con 0x7fe2600b86f0 2024-04-27T07:05:29.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25f59e640 1 --2- 172.21.15.12:0/2356725424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b86f0 0x7fe2600b8b20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe260071c50 tx=0x7fe25801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3b9cc01e72e658 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25dd9b640 1 -- 172.21.15.12:0/2356725424 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260145d40 con 0x7fe2600b86f0 2024-04-27T07:05:29.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25dd9b640 1 -- 172.21.15.12:0/2356725424 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2580a55f0 con 0x7fe2600b86f0 2024-04-27T07:05:29.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.316+0000 7fe25dd9b640 1 -- 172.21.15.12:0/2356725424 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580a59c0 con 0x7fe2600b86f0 2024-04-27T07:05:29.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- 172.21.15.12:0/2356725424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b86f0 msgr2=0x7fe2600b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 --2- 172.21.15.12:0/2356725424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b86f0 0x7fe2600b8b20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe260071c50 tx=0x7fe25801c920 comp rx=0 tx=0).stop 2024-04-27T07:05:29.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- 172.21.15.12:0/2356725424 shutdown_connections 2024-04-27T07:05:29.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 --2- 172.21.15.12:0/2356725424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b86f0 0x7fe2600b8b20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 --2- 172.21.15.12:0/2356725424 >> v2:172.21.15.53:3300/0 conn(0x7fe2600b9f20 0x7fe2600b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- 172.21.15.12:0/2356725424 >> 172.21.15.12:0/2356725424 conn(0x7fe2600ba740 msgr2=0x7fe2600bab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:29.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- 172.21.15.12:0/2356725424 shutdown_connections 2024-04-27T07:05:29.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- 172.21.15.12:0/2356725424 wait complete. 2024-04-27T07:05:29.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 Processor -- start 2024-04-27T07:05:29.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- start start 2024-04-27T07:05:29.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2600b7430 0x7fe2600b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe2600b86f0 0x7fe26009c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b9f20 0x7fe26009bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe26006c7a0 con 0x7fe2600b9f20 2024-04-27T07:05:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe26006c620 con 0x7fe2600b7430 2024-04-27T07:05:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe26006c920 con 0x7fe2600b86f0 2024-04-27T07:05:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25f59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b9f20 0x7fe26009bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25f59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b9f20 0x7fe26009bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55658/0 (socket says 172.21.15.12:55658) 2024-04-27T07:05:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25f59e640 1 -- 172.21.15.12:0/3754385065 learned_addr learned my addr 172.21.15.12:0/3754385065 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25ed9d640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2600b7430 0x7fe2600b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25e59c640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe2600b86f0 0x7fe26009c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25f59e640 1 -- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe2600b86f0 msgr2=0x7fe26009c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25f59e640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe2600b86f0 0x7fe26009c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25f59e640 1 -- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2600b7430 msgr2=0x7fe2600b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25f59e640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2600b7430 0x7fe2600b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25f59e640 1 -- 172.21.15.12:0/3754385065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26009d2f0 con 0x7fe2600b9f20 2024-04-27T07:05:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe25f59e640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b9f20 0x7fe26009bb50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe25801a550 tx=0x7fe2580196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe2577fe640 1 -- 172.21.15.12:0/3754385065 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580bc020 con 0x7fe2600b9f20 2024-04-27T07:05:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe2577fe640 1 -- 172.21.15.12:0/3754385065 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2580b8670 con 0x7fe2600b9f20 2024-04-27T07:05:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- 172.21.15.12:0/3754385065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe260148d90 con 0x7fe2600b9f20 2024-04-27T07:05:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe2577fe640 1 -- 172.21.15.12:0/3754385065 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580b8a60 con 0x7fe2600b9f20 2024-04-27T07:05:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.320+0000 7fe265091640 1 -- 172.21.15.12:0/3754385065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26009cd20 con 0x7fe2600b9f20 2024-04-27T07:05:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.324+0000 7fe265091640 1 -- 172.21.15.12:0/3754385065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26009d040 con 0x7fe2600b9f20 2024-04-27T07:05:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.324+0000 7fe2577fe640 1 -- 172.21.15.12:0/3754385065 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe2580bf020 con 0x7fe2600b9f20 2024-04-27T07:05:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.324+0000 7fe2577fe640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe240068380 0x7fe24006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.324+0000 7fe2577fe640 1 -- 172.21.15.12:0/3754385065 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fe258121790 con 0x7fe2600b9f20 2024-04-27T07:05:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.324+0000 7fe25ed9d640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe240068380 0x7fe24006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.324+0000 7fe25ed9d640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe240068380 0x7fe24006a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe248001600 tx=0x7fe248009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.328+0000 7fe2577fe640 1 -- 172.21.15.12:0/3754385065 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe2580c6030 con 0x7fe2600b9f20 2024-04-27T07:05:29.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.488+0000 7fe265091640 1 -- 172.21.15.12:0/3754385065 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe260148b80 con 0x7fe240068380 2024-04-27T07:05:29.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe2577fe640 1 -- 172.21.15.12:0/3754385065 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe260148b80 con 0x7fe240068380 2024-04-27T07:05:29.497 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 3 to deep-scrub 2024-04-27T07:05:29.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 -- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe240068380 msgr2=0x7fe24006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe240068380 0x7fe24006a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe248001600 tx=0x7fe248009040 comp rx=0 tx=0).stop 2024-04-27T07:05:29.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 -- 172.21.15.12:0/3754385065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b9f20 msgr2=0x7fe26009bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b9f20 0x7fe26009bb50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe25801a550 tx=0x7fe2580196c0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 -- 172.21.15.12:0/3754385065 shutdown_connections 2024-04-27T07:05:29.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe240068380 0x7fe24006a840 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2600b9f20 0x7fe26009bb50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe2600b86f0 0x7fe26009c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 --2- 172.21.15.12:0/3754385065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2600b7430 0x7fe2600b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 -- 172.21.15.12:0/3754385065 >> 172.21.15.12:0/3754385065 conn(0x7fe2600ba740 msgr2=0x7fe260074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:29.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 -- 172.21.15.12:0/3754385065 shutdown_connections 2024-04-27T07:05:29.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.492+0000 7fe265091640 1 -- 172.21.15.12:0/3754385065 wait complete. 2024-04-27T07:05:29.515 INFO:tasks.ceph:Scrubbing osd.4 2024-04-27T07:05:29.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:05:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.640+0000 7fe2f4b0f640 1 Processor -- start 2024-04-27T07:05:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.640+0000 7fe2f4b0f640 1 -- start start 2024-04-27T07:05:29.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2f4b0f640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fe2f009ce40 0x7fe2f006da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2f4b0f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f0149ff0 con 0x7fe2f009e720 2024-04-27T07:05:29.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2f4b0f640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f0145fb0 con 0x7fe2f009ca60 2024-04-27T07:05:29.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2f4b0f640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f006df60 con 0x7fe2f009ce40 2024-04-27T07:05:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2edd9b640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe2f009e720 0x7fe2f01498e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52246/0 (socket says 172.21.15.12:52246) 2024-04-27T07:05:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2edd9b640 1 -- 172.21.15.12:0/1709175558 learned_addr learned my addr 172.21.15.12:0/1709175558 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2eed9d640 1 --2- 172.21.15.12:0/1709175558 >> v2:172.21.15.53:3300/0 conn(0x7fe2f009ce40 0x7fe2f006da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2ed59a640 1 -- 172.21.15.12:0/1709175558 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1212101297 0 0) 0x7fe2f0149ff0 con 0x7fe2f009e720 2024-04-27T07:05:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2ed59a640 1 -- 172.21.15.12:0/1709175558 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2dc003530 con 0x7fe2f009e720 2024-04-27T07:05:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2eed9d640 1 -- 172.21.15.12:0/1709175558 >> v1:172.21.15.53:6789/0 conn(0x7fe2f009ca60 legacy=0x7fe2f01491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2eed9d640 1 -- 172.21.15.12:0/1709175558 >> v1:172.21.15.12:6789/0 conn(0x7fe2f009e720 legacy=0x7fe2f01498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.644+0000 7fe2eed9d640 1 -- 172.21.15.12:0/1709175558 --> v2:172.21.15.53:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f009bce0 con 0x7fe2f009ce40 2024-04-27T07:05:29.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2eed9d640 1 --2- 172.21.15.12:0/1709175558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2f009ce40 0x7fe2f006da20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80097c0 tx=0x7fe2e801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4bdf6f4d44c3cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2ed59a640 1 -- 172.21.15.12:0/1709175558 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80b1070 con 0x7fe2f009ce40 2024-04-27T07:05:29.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2ed59a640 1 -- 172.21.15.12:0/1709175558 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e80ac5f0 con 0x7fe2f009ce40 2024-04-27T07:05:29.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2ed59a640 1 -- 172.21.15.12:0/1709175558 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80ac9a0 con 0x7fe2f009ce40 2024-04-27T07:05:29.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1709175558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2f009ce40 msgr2=0x7fe2f006da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 --2- 172.21.15.12:0/1709175558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2f009ce40 0x7fe2f006da20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80097c0 tx=0x7fe2e801c4a0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1709175558 shutdown_connections 2024-04-27T07:05:29.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 --2- 172.21.15.12:0/1709175558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2f009ce40 0x7fe2f006da20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1709175558 >> 172.21.15.12:0/1709175558 conn(0x7fe2f00b9840 msgr2=0x7fe2f00b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:29.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1709175558 shutdown_connections 2024-04-27T07:05:29.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1709175558 wait complete. 2024-04-27T07:05:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 Processor -- start 2024-04-27T07:05:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- start start 2024-04-27T07:05:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2f009ca60 0x7fe2f0149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f009ce40 0x7fe2f014f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe2f009e720 0x7fe2f014ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f0153eb0 con 0x7fe2f009ce40 2024-04-27T07:05:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f0153d30 con 0x7fe2f009ca60 2024-04-27T07:05:29.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f0154030 con 0x7fe2f009e720 2024-04-27T07:05:29.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2edd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f009ce40 0x7fe2f014f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2edd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f009ce40 0x7fe2f014f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55666/0 (socket says 172.21.15.12:55666) 2024-04-27T07:05:29.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2edd9b640 1 -- 172.21.15.12:0/1935373354 learned_addr learned my addr 172.21.15.12:0/1935373354 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:29.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2ee59c640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2f009ca60 0x7fe2f0149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2eed9d640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe2f009e720 0x7fe2f014ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2edd9b640 1 -- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe2f009e720 msgr2=0x7fe2f014ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2edd9b640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe2f009e720 0x7fe2f014ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2edd9b640 1 -- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2f009ca60 msgr2=0x7fe2f0149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2edd9b640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe2f009ca60 0x7fe2f0149600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2edd9b640 1 -- 172.21.15.12:0/1935373354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f0150b90 con 0x7fe2f009ce40 2024-04-27T07:05:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2eed9d640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fe2f009e720 0x7fe2f014ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2edd9b640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f009ce40 0x7fe2f014f880 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800cbc0 tx=0x7fe2d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2e6ffd640 1 -- 172.21.15.12:0/1935373354 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8017070 con 0x7fe2f009ce40 2024-04-27T07:05:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2e6ffd640 1 -- 172.21.15.12:0/1935373354 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d800ad40 con 0x7fe2f009ce40 2024-04-27T07:05:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2e6ffd640 1 -- 172.21.15.12:0/1935373354 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8012600 con 0x7fe2f009ce40 2024-04-27T07:05:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f014e690 con 0x7fe2f009ce40 2024-04-27T07:05:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.648+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f01e01d0 con 0x7fe2f009ce40 2024-04-27T07:05:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2f01e04f0 con 0x7fe2f009ce40 2024-04-27T07:05:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2e6ffd640 1 -- 172.21.15.12:0/1935373354 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe2d801a070 con 0x7fe2f009ce40 2024-04-27T07:05:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2e6ffd640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe2d0068380 0x7fe2d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2e6ffd640 1 -- 172.21.15.12:0/1935373354 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fe2d8086c40 con 0x7fe2f009ce40 2024-04-27T07:05:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2e6ffd640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] conn(0x7fe2d006e240 0x7fe2d0070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2e6ffd640 1 -- 172.21.15.12:0/1935373354 --> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2d8016ec0 con 0x7fe2d006e240 2024-04-27T07:05:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2e6ffd640 1 -- 172.21.15.12:0/1935373354 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe2d8086ff0 con 0x7fe2f009ce40 2024-04-27T07:05:29.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2ee59c640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe2d0068380 0x7fe2d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2eed9d640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] conn(0x7fe2d006e240 0x7fe2d0070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2ee59c640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe2d0068380 0x7fe2d006a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0000f70 tx=0x7fe2e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.652+0000 7fe2eed9d640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] conn(0x7fe2d006e240 0x7fe2d0070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.656+0000 7fe2e6ffd640 1 -- 172.21.15.12:0/1935373354 <== osd.4 v2:172.21.15.53:6800/151807054 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2d8016ec0 con 0x7fe2d006e240 2024-04-27T07:05:29.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.668+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 --> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe2f0068d00 con 0x7fe2d006e240 2024-04-27T07:05:29.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.668+0000 7fe2e6ffd640 1 -- 172.21.15.12:0/1935373354 <== osd.4 v2:172.21.15.53:6800/151807054 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe2f0068d00 con 0x7fe2d006e240 2024-04-27T07:05:29.676 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:05:29.676 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:05:29.677 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:05:29.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.668+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] conn(0x7fe2d006e240 msgr2=0x7fe2d0070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.668+0000 7fe2f4b0f640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:6800/151807054,v1:172.21.15.53:6801/151807054] conn(0x7fe2d006e240 0x7fe2d0070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.668+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe2d0068380 msgr2=0x7fe2d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.672+0000 7fe2f4b0f640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fe2d0068380 0x7fe2d006a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0000f70 tx=0x7fe2e0009040 comp rx=0 tx=0).stop 2024-04-27T07:05:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.672+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f009ce40 msgr2=0x7fe2f014f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.672+0000 7fe2f4b0f640 1 --2- 172.21.15.12:0/1935373354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f009ce40 0x7fe2f014f880 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800cbc0 tx=0x7fe2d800a430 comp rx=0 tx=0).stop 2024-04-27T07:05:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.672+0000 7fe2eed9d640 1 -- 172.21.15.12:0/1935373354 reap_dead start 2024-04-27T07:05:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.672+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 shutdown_connections 2024-04-27T07:05:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.672+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 >> 172.21.15.12:0/1935373354 conn(0x7fe2f00b9840 msgr2=0x7fe2f01454c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:29.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.672+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 shutdown_connections 2024-04-27T07:05:29.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.672+0000 7fe2f4b0f640 1 -- 172.21.15.12:0/1935373354 wait complete. 2024-04-27T07:05:29.692 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-27T07:05:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa9029640 1 Processor -- start 2024-04-27T07:05:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa9029640 1 -- start start 2024-04-27T07:05:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa9029640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1fa40b3db0 0x7f1fa409da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa9029640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f1fa406e410 0x7f1fa409dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa9029640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa4144cb0 con 0x7f1fa40b4ae0 2024-04-27T07:05:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa9029640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa4144a80 con 0x7f1fa40b3db0 2024-04-27T07:05:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa9029640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa40b4ec0 con 0x7f1fa406e410 2024-04-27T07:05:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1fa40b3db0 0x7f1fa409da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1fa40b3db0 0x7f1fa409da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55684/0 (socket says 172.21.15.12:55684) 2024-04-27T07:05:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa259c640 1 -- 172.21.15.12:0/3156312946 learned_addr learned my addr 172.21.15.12:0/3156312946 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa259c640 1 -- 172.21.15.12:0/3156312946 >> v2:172.21.15.53:3300/0 conn(0x7f1fa406e410 msgr2=0x7f1fa409dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa259c640 1 --2- 172.21.15.12:0/3156312946 >> v2:172.21.15.53:3300/0 conn(0x7f1fa406e410 0x7f1fa409dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa259c640 1 -- 172.21.15.12:0/3156312946 >> v1:172.21.15.53:6789/0 conn(0x7f1fa40b4ae0 legacy=0x7f1fa41454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.804+0000 7f1fa259c640 1 -- 172.21.15.12:0/3156312946 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa4144f90 con 0x7f1fa40b3db0 2024-04-27T07:05:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa259c640 1 --2- 172.21.15.12:0/3156312946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 0x7f1fa409da80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4149090 tx=0x7f1f9801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a844ef7dea8188a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa1d9b640 1 -- 172.21.15.12:0/3156312946 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f980ab610 con 0x7f1fa40b3db0 2024-04-27T07:05:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa1d9b640 1 -- 172.21.15.12:0/3156312946 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f980b0070 con 0x7f1fa40b3db0 2024-04-27T07:05:29.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa1d9b640 1 -- 172.21.15.12:0/3156312946 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f980ab9d0 con 0x7f1fa40b3db0 2024-04-27T07:05:29.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 -- 172.21.15.12:0/3156312946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 msgr2=0x7f1fa409da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 --2- 172.21.15.12:0/3156312946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 0x7f1fa409da80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4149090 tx=0x7f1f9801c500 comp rx=0 tx=0).stop 2024-04-27T07:05:29.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 -- 172.21.15.12:0/3156312946 shutdown_connections 2024-04-27T07:05:29.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 --2- 172.21.15.12:0/3156312946 >> v2:172.21.15.53:3300/0 conn(0x7f1fa406e410 0x7f1fa409dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 --2- 172.21.15.12:0/3156312946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 0x7f1fa409da80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 -- 172.21.15.12:0/3156312946 >> 172.21.15.12:0/3156312946 conn(0x7f1fa40b93b0 msgr2=0x7f1fa40b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:29.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 -- 172.21.15.12:0/3156312946 shutdown_connections 2024-04-27T07:05:29.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 -- 172.21.15.12:0/3156312946 wait complete. 2024-04-27T07:05:29.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 Processor -- start 2024-04-27T07:05:29.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 -- start start 2024-04-27T07:05:29.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa406e410 0x7f1fa4070be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 0x7f1fa4144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1fa40b4ae0 0x7f1fa40b8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4151ef0 con 0x7f1fa40b3db0 2024-04-27T07:05:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4151d70 con 0x7f1fa406e410 2024-04-27T07:05:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa9029640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa4152070 con 0x7f1fa40b4ae0 2024-04-27T07:05:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 0x7f1fa4144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 0x7f1fa4144010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55686/0 (socket says 172.21.15.12:55686) 2024-04-27T07:05:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.808+0000 7f1fa259c640 1 -- 172.21.15.12:0/475554372 learned_addr learned my addr 172.21.15.12:0/475554372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:29.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa259c640 1 -- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1fa40b4ae0 msgr2=0x7f1fa40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa359e640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1fa40b4ae0 0x7f1fa40b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa2d9d640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa406e410 0x7f1fa4070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa259c640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1fa40b4ae0 0x7f1fa40b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa259c640 1 -- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa406e410 msgr2=0x7f1fa4070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa259c640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa406e410 0x7f1fa4070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa259c640 1 -- 172.21.15.12:0/475554372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa4072aa0 con 0x7f1fa40b3db0 2024-04-27T07:05:29.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa259c640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 0x7f1fa4144010 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1f980ad040 tx=0x7f1f9801ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1f8b7fe640 1 -- 172.21.15.12:0/475554372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f980b48c0 con 0x7f1fa40b3db0 2024-04-27T07:05:29.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1f8b7fe640 1 -- 172.21.15.12:0/475554372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f980b4a60 con 0x7f1fa40b3db0 2024-04-27T07:05:29.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa9029640 1 -- 172.21.15.12:0/475554372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa4071dc0 con 0x7f1fa40b3db0 2024-04-27T07:05:29.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1f8b7fe640 1 -- 172.21.15.12:0/475554372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f980b4e50 con 0x7f1fa40b3db0 2024-04-27T07:05:29.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa9029640 1 -- 172.21.15.12:0/475554372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa4073370 con 0x7f1fa40b3db0 2024-04-27T07:05:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa9029640 1 -- 172.21.15.12:0/475554372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa4068c20 con 0x7f1fa40b3db0 2024-04-27T07:05:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1f8b7fe640 1 -- 172.21.15.12:0/475554372 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1f980c7390 con 0x7f1fa40b3db0 2024-04-27T07:05:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1f8b7fe640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1f90068520 0x7f1f9006a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1f8b7fe640 1 -- 172.21.15.12:0/475554372 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f1f98121010 con 0x7f1fa40b3db0 2024-04-27T07:05:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa2d9d640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1f90068520 0x7f1f9006a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.812+0000 7f1fa2d9d640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1f90068520 0x7f1f9006a9e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c007dc0 tx=0x7f1f9c0072f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:29.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.816+0000 7f1f8b7fe640 1 -- 172.21.15.12:0/475554372 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1f980c4060 con 0x7f1fa40b3db0 2024-04-27T07:05:29.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 -- 172.21.15.12:0/475554372 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f1fa4069550 con 0x7f1f90068520 2024-04-27T07:05:29.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1f8b7fe640 1 -- 172.21.15.12:0/475554372 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1fa4069550 con 0x7f1f90068520 2024-04-27T07:05:29.985 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 4 to deep-scrub 2024-04-27T07:05:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 -- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1f90068520 msgr2=0x7f1f9006a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1f90068520 0x7f1f9006a9e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c007dc0 tx=0x7f1f9c0072f0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 -- 172.21.15.12:0/475554372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 msgr2=0x7f1fa4144010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 0x7f1fa4144010 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1f980ad040 tx=0x7f1f9801ad40 comp rx=0 tx=0).stop 2024-04-27T07:05:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 -- 172.21.15.12:0/475554372 shutdown_connections 2024-04-27T07:05:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f1f90068520 0x7f1f9006a9e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f1fa40b4ae0 0x7f1fa40b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fa40b3db0 0x7f1fa4144010 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 --2- 172.21.15.12:0/475554372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fa406e410 0x7f1fa4070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 -- 172.21.15.12:0/475554372 >> 172.21.15.12:0/475554372 conn(0x7f1fa40b93b0 msgr2=0x7f1fa4141e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 -- 172.21.15.12:0/475554372 shutdown_connections 2024-04-27T07:05:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:29.980+0000 7f1fa9029640 1 -- 172.21.15.12:0/475554372 wait complete. 2024-04-27T07:05:30.005 INFO:tasks.ceph:Scrubbing osd.5 2024-04-27T07:05:30.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:05:30.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba50d46640 1 Processor -- start 2024-04-27T07:05:30.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba50d46640 1 -- start start 2024-04-27T07:05:30.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba50d46640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fba4c09ca60 0x7fba4c06da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba50d46640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba4c149ff0 con 0x7fba4c09ce40 2024-04-27T07:05:30.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba50d46640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba4c145fb0 con 0x7fba4c09e720 2024-04-27T07:05:30.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba50d46640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fba4c06df60 con 0x7fba4c09ca60 2024-04-27T07:05:30.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba4a59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fba4c09ca60 0x7fba4c06da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba4a59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fba4c09ca60 0x7fba4c06da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55692/0 (socket says 172.21.15.12:55692) 2024-04-27T07:05:30.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba4ad9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fba4c09ce40 0x7fba4c1498e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52250/0 (socket says 172.21.15.12:52250) 2024-04-27T07:05:30.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba4a59c640 1 -- 172.21.15.12:0/2276184647 learned_addr learned my addr 172.21.15.12:0/2276184647 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:30.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba4a59c640 1 -- 172.21.15.12:0/2276184647 >> v1:172.21.15.53:6789/0 conn(0x7fba4c09e720 legacy=0x7fba4c1491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:30.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba4a59c640 1 -- 172.21.15.12:0/2276184647 >> v1:172.21.15.12:6789/0 conn(0x7fba4c09ce40 legacy=0x7fba4c1498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.120+0000 7fba4a59c640 1 -- 172.21.15.12:0/2276184647 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c09bce0 con 0x7fba4c09ca60 2024-04-27T07:05:30.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba4a59c640 1 --2- 172.21.15.12:0/2276184647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba4c09ca60 0x7fba4c06da20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fba44009e50 tx=0x7fba4401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d6145b524e750a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba4959a640 1 -- 172.21.15.12:0/2276184647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba440a56c0 con 0x7fba4c09ca60 2024-04-27T07:05:30.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba4959a640 1 -- 172.21.15.12:0/2276184647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba440a5860 con 0x7fba4c09ca60 2024-04-27T07:05:30.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba4959a640 1 -- 172.21.15.12:0/2276184647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba440a5c30 con 0x7fba4c09ca60 2024-04-27T07:05:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba50d46640 1 -- 172.21.15.12:0/2276184647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba4c09ca60 msgr2=0x7fba4c06da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba50d46640 1 --2- 172.21.15.12:0/2276184647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba4c09ca60 0x7fba4c06da20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fba44009e50 tx=0x7fba4401c9f0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba50d46640 1 -- 172.21.15.12:0/2276184647 shutdown_connections 2024-04-27T07:05:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba50d46640 1 --2- 172.21.15.12:0/2276184647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba4c09ca60 0x7fba4c06da20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba50d46640 1 -- 172.21.15.12:0/2276184647 >> 172.21.15.12:0/2276184647 conn(0x7fba4c0b9840 msgr2=0x7fba4c0b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba50d46640 1 -- 172.21.15.12:0/2276184647 shutdown_connections 2024-04-27T07:05:30.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba50d46640 1 -- 172.21.15.12:0/2276184647 wait complete. 2024-04-27T07:05:30.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba50d46640 1 Processor -- start 2024-04-27T07:05:30.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.124+0000 7fba50d46640 1 -- start start 2024-04-27T07:05:30.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba50d46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba4c09ca60 0x7fba4c141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba50d46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c09ce40 0x7fba4c143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba50d46640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba4c09e720 0x7fba4c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba50d46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c153eb0 con 0x7fba4c09ca60 2024-04-27T07:05:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba50d46640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fba4c153d30 con 0x7fba4c09ce40 2024-04-27T07:05:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba50d46640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fba4c154030 con 0x7fba4c09e720 2024-04-27T07:05:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba4a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba4c09ca60 0x7fba4c141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba4a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba4c09ca60 0x7fba4c141420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55694/0 (socket says 172.21.15.12:55694) 2024-04-27T07:05:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba4a59c640 1 -- 172.21.15.12:0/2181348932 learned_addr learned my addr 172.21.15.12:0/2181348932 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba49d9b640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c09ce40 0x7fba4c143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba4ad9d640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba4c09e720 0x7fba4c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba4a59c640 1 -- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba4c09e720 msgr2=0x7fba4c0b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba4a59c640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fba4c09e720 0x7fba4c0b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba4a59c640 1 -- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c09ce40 msgr2=0x7fba4c143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba4a59c640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba4c09ce40 0x7fba4c143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba4a59c640 1 -- 172.21.15.12:0/2181348932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c0b5440 con 0x7fba4c09ca60 2024-04-27T07:05:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba4a59c640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba4c09ca60 0x7fba4c141420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fba44019600 tx=0x7fba44019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba3affd640 1 -- 172.21.15.12:0/2181348932 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba440bc020 con 0x7fba4c09ca60 2024-04-27T07:05:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba3affd640 1 -- 172.21.15.12:0/2181348932 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba440b8670 con 0x7fba4c09ca60 2024-04-27T07:05:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4c073880 con 0x7fba4c09ca60 2024-04-27T07:05:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba3affd640 1 -- 172.21.15.12:0/2181348932 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba440b8a20 con 0x7fba4c09ca60 2024-04-27T07:05:30.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4c144990 con 0x7fba4c09ca60 2024-04-27T07:05:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba4c068d00 con 0x7fba4c09ca60 2024-04-27T07:05:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba3affd640 1 -- 172.21.15.12:0/2181348932 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fba440bf020 con 0x7fba4c09ca60 2024-04-27T07:05:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba3affd640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fba2c068380 0x7fba2c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba3affd640 1 -- 172.21.15.12:0/2181348932 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fba44121bd0 con 0x7fba4c09ca60 2024-04-27T07:05:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba3affd640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] conn(0x7fba2c06e240 0x7fba2c070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba3affd640 1 -- 172.21.15.12:0/2181348932 --> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba440c0e70 con 0x7fba2c06e240 2024-04-27T07:05:30.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba3affd640 1 -- 172.21.15.12:0/2181348932 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fba44122060 con 0x7fba4c09ca60 2024-04-27T07:05:30.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.128+0000 7fba49d9b640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fba2c068380 0x7fba2c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.132+0000 7fba4ad9d640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] conn(0x7fba2c06e240 0x7fba2c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.132+0000 7fba49d9b640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fba2c068380 0x7fba2c06a840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fba4c1481d0 tx=0x7fba40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.132+0000 7fba4ad9d640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] conn(0x7fba2c06e240 0x7fba2c070660 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.132+0000 7fba3affd640 1 -- 172.21.15.12:0/2181348932 <== osd.5 v2:172.21.15.53:6824/2061790692 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba440c0e70 con 0x7fba2c06e240 2024-04-27T07:05:30.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.144+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 --> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fba4c0b3830 con 0x7fba2c06e240 2024-04-27T07:05:30.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba3affd640 1 -- 172.21.15.12:0/2181348932 <== osd.5 v2:172.21.15.53:6824/2061790692 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fba4c0b3830 con 0x7fba2c06e240 2024-04-27T07:05:30.153 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:05:30.154 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:05:30.154 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:05:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] conn(0x7fba2c06e240 msgr2=0x7fba2c070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba50d46640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:6824/2061790692,v1:172.21.15.53:6825/2061790692] conn(0x7fba2c06e240 0x7fba2c070660 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fba2c068380 msgr2=0x7fba2c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba50d46640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fba2c068380 0x7fba2c06a840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fba4c1481d0 tx=0x7fba40008040 comp rx=0 tx=0).stop 2024-04-27T07:05:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba4c09ca60 msgr2=0x7fba4c141420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba50d46640 1 --2- 172.21.15.12:0/2181348932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fba4c09ca60 0x7fba4c141420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fba44019600 tx=0x7fba44019790 comp rx=0 tx=0).stop 2024-04-27T07:05:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba4ad9d640 1 -- 172.21.15.12:0/2181348932 reap_dead start 2024-04-27T07:05:30.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 shutdown_connections 2024-04-27T07:05:30.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 >> 172.21.15.12:0/2181348932 conn(0x7fba4c0b9840 msgr2=0x7fba4c1484a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:30.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 shutdown_connections 2024-04-27T07:05:30.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.148+0000 7fba50d46640 1 -- 172.21.15.12:0/2181348932 wait complete. 2024-04-27T07:05:30.168 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-27T07:05:30.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f871b8f2640 1 Processor -- start 2024-04-27T07:05:30.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f871b8f2640 1 -- start start 2024-04-27T07:05:30.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f871b8f2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f87140b7880 0x7f87140b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f871b8f2640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87140bc3b0 con 0x7f87140bbfd0 2024-04-27T07:05:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f871b8f2640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87140b7d10 con 0x7f87140b2b60 2024-04-27T07:05:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f871b8f2640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f87140b63f0 con 0x7f87140b7880 2024-04-27T07:05:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f8718e8d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f87140b7880 0x7f87140b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f871968e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f87140bbfd0 0x7f871413fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52264/0 (socket says 172.21.15.12:52264) 2024-04-27T07:05:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f871968e640 1 -- 172.21.15.12:0/3867071353 learned_addr learned my addr 172.21.15.12:0/3867071353 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f8718e8d640 1 -- 172.21.15.12:0/3867071353 >> v1:172.21.15.53:6789/0 conn(0x7f87140b2b60 legacy=0x7f87141503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f8718e8d640 1 -- 172.21.15.12:0/3867071353 >> v1:172.21.15.12:6789/0 conn(0x7f87140bbfd0 legacy=0x7f871413fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f8718e8d640 1 -- 172.21.15.12:0/3867071353 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87140b6640 con 0x7f87140b7880 2024-04-27T07:05:30.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.276+0000 7f870bfff640 1 -- 172.21.15.12:0/3867071353 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185058470 0 0) 0x7f87140bc3b0 con 0x7f87140bbfd0 2024-04-27T07:05:30.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8718e8d640 1 --2- 172.21.15.12:0/3867071353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87140b7880 0x7f87140b42d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f87040097c0 tx=0x7f870401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9135410239ab3111 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f870bfff640 1 -- 172.21.15.12:0/3867071353 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87040b0070 con 0x7f87140b7880 2024-04-27T07:05:30.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f870bfff640 1 -- 172.21.15.12:0/3867071353 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87040ab600 con 0x7f87140b7880 2024-04-27T07:05:30.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f870bfff640 1 -- 172.21.15.12:0/3867071353 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87040ab9d0 con 0x7f87140b7880 2024-04-27T07:05:30.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 -- 172.21.15.12:0/3867071353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87140b7880 msgr2=0x7f87140b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 --2- 172.21.15.12:0/3867071353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87140b7880 0x7f87140b42d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f87040097c0 tx=0x7f870401c460 comp rx=0 tx=0).stop 2024-04-27T07:05:30.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 -- 172.21.15.12:0/3867071353 shutdown_connections 2024-04-27T07:05:30.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 --2- 172.21.15.12:0/3867071353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87140b7880 0x7f87140b42d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 -- 172.21.15.12:0/3867071353 >> 172.21.15.12:0/3867071353 conn(0x7f8714068040 msgr2=0x7f87140bb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:30.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 -- 172.21.15.12:0/3867071353 shutdown_connections 2024-04-27T07:05:30.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 -- 172.21.15.12:0/3867071353 wait complete. 2024-04-27T07:05:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 Processor -- start 2024-04-27T07:05:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 -- start start 2024-04-27T07:05:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87140b2b60 0x7f8714145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87140bbfd0 0x7f87141e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8714145da0 0x7f87141e22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f87141546f0 con 0x7f8714145da0 2024-04-27T07:05:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8714140240 con 0x7f87140b2b60 2024-04-27T07:05:30.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871b8f2640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8714154870 con 0x7f87140bbfd0 2024-04-27T07:05:30.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8719e8f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8714145da0 0x7f87141e22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8719e8f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8714145da0 0x7f87141e22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55716/0 (socket says 172.21.15.12:55716) 2024-04-27T07:05:30.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8719e8f640 1 -- 172.21.15.12:0/120846565 learned_addr learned my addr 172.21.15.12:0/120846565 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:30.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8718e8d640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87140bbfd0 0x7f87141e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f871968e640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87140b2b60 0x7f8714145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8719e8f640 1 -- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87140bbfd0 msgr2=0x7f87141e1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8719e8f640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87140bbfd0 0x7f87141e1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8719e8f640 1 -- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87140b2b60 msgr2=0x7f8714145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8719e8f640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87140b2b60 0x7f8714145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8719e8f640 1 -- 172.21.15.12:0/120846565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871413c700 con 0x7f8714145da0 2024-04-27T07:05:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.280+0000 7f8719e8f640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8714145da0 0x7f87141e22d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c780 tx=0x7f870c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f8709ffb640 1 -- 172.21.15.12:0/120846565 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c016020 con 0x7f8714145da0 2024-04-27T07:05:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f8709ffb640 1 -- 172.21.15.12:0/120846565 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870c00a9f0 con 0x7f8714145da0 2024-04-27T07:05:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f871b8f2640 1 -- 172.21.15.12:0/120846565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87141449e0 con 0x7f8714145da0 2024-04-27T07:05:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f8709ffb640 1 -- 172.21.15.12:0/120846565 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c00ade0 con 0x7f8714145da0 2024-04-27T07:05:30.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f871b8f2640 1 -- 172.21.15.12:0/120846565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87141446d0 con 0x7f8714145da0 2024-04-27T07:05:30.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f871b8f2640 1 -- 172.21.15.12:0/120846565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87141451b0 con 0x7f8714145da0 2024-04-27T07:05:30.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f8709ffb640 1 -- 172.21.15.12:0/120846565 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f870c019020 con 0x7f8714145da0 2024-04-27T07:05:30.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f8709ffb640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f86f8068380 0x7f86f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f8709ffb640 1 -- 172.21.15.12:0/120846565 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f870c086560 con 0x7f8714145da0 2024-04-27T07:05:30.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f871968e640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f86f8068380 0x7f86f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.284+0000 7f871968e640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f86f8068380 0x7f86f806a840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8710000f70 tx=0x7f8710007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.288+0000 7f8709ffb640 1 -- 172.21.15.12:0/120846565 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f870c020030 con 0x7f8714145da0 2024-04-27T07:05:30.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.448+0000 7f871b8f2640 1 -- 172.21.15.12:0/120846565 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f87140babd0 con 0x7f86f8068380 2024-04-27T07:05:30.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.448+0000 7f8709ffb640 1 -- 172.21.15.12:0/120846565 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f87140babd0 con 0x7f86f8068380 2024-04-27T07:05:30.456 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 5 to deep-scrub 2024-04-27T07:05:30.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 -- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f86f8068380 msgr2=0x7f86f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f86f8068380 0x7f86f806a840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8710000f70 tx=0x7f8710007c10 comp rx=0 tx=0).stop 2024-04-27T07:05:30.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 -- 172.21.15.12:0/120846565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8714145da0 msgr2=0x7f87141e22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8714145da0 0x7f87141e22d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c780 tx=0x7f870c00cc40 comp rx=0 tx=0).stop 2024-04-27T07:05:30.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 -- 172.21.15.12:0/120846565 shutdown_connections 2024-04-27T07:05:30.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f86f8068380 0x7f86f806a840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8714145da0 0x7f87141e22d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f87140bbfd0 0x7f87141e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 --2- 172.21.15.12:0/120846565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f87140b2b60 0x7f8714145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 -- 172.21.15.12:0/120846565 >> 172.21.15.12:0/120846565 conn(0x7f8714068040 msgr2=0x7f87140b80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:30.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 -- 172.21.15.12:0/120846565 shutdown_connections 2024-04-27T07:05:30.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.452+0000 7f871b8f2640 1 -- 172.21.15.12:0/120846565 wait complete. 2024-04-27T07:05:30.475 INFO:tasks.ceph:Scrubbing osd.6 2024-04-27T07:05:30.476 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:05:30.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.600+0000 7f806f234640 1 Processor -- start 2024-04-27T07:05:30.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.600+0000 7f806f234640 1 -- start start 2024-04-27T07:05:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f806f234640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f806809ca60 0x7f806806da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f806f234640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f806809e720 0x7f80681491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f806f234640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f806809ce40 0x7f8068149710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f806f234640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f806806df60 con 0x7f806809e720 2024-04-27T07:05:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f806f234640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f806809e320 con 0x7f806809ce40 2024-04-27T07:05:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f806f234640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f806809bce0 con 0x7f806809ca60 2024-04-27T07:05:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f8067fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f806809e720 0x7f80681491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f8067fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f806809e720 0x7f80681491d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55730/0 (socket says 172.21.15.12:55730) 2024-04-27T07:05:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f8067fff640 1 -- 172.21.15.12:0/2893837064 learned_addr learned my addr 172.21.15.12:0/2893837064 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f806cfd0640 1 --2- 172.21.15.12:0/2893837064 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f806809ca60 0x7f806806da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f806d7d1640 1 --2- 172.21.15.12:0/2893837064 >> v2:172.21.15.53:3300/0 conn(0x7f806809ce40 0x7f8068149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f8067fff640 1 -- 172.21.15.12:0/2893837064 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f806809ca60 msgr2=0x7f806806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:30.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f8067fff640 1 --2- 172.21.15.12:0/2893837064 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f806809ca60 0x7f806806da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f8067fff640 1 -- 172.21.15.12:0/2893837064 >> v2:172.21.15.53:3300/0 conn(0x7f806809ce40 msgr2=0x7f8068149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:30.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f8067fff640 1 --2- 172.21.15.12:0/2893837064 >> v2:172.21.15.53:3300/0 conn(0x7f806809ce40 0x7f8068149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f8067fff640 1 -- 172.21.15.12:0/2893837064 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806809bfa0 con 0x7f806809e720 2024-04-27T07:05:30.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f8067fff640 1 --2- 172.21.15.12:0/2893837064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806809e720 0x7f80681491d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f806809eba0 tx=0x7f805c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09c96f96e88b718 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f80677fe640 1 -- 172.21.15.12:0/2893837064 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0b1070 con 0x7f806809e720 2024-04-27T07:05:30.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f80677fe640 1 -- 172.21.15.12:0/2893837064 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f805c0ac420 con 0x7f806809e720 2024-04-27T07:05:30.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.604+0000 7f80677fe640 1 -- 172.21.15.12:0/2893837064 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0ac7f0 con 0x7f806809e720 2024-04-27T07:05:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- 172.21.15.12:0/2893837064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806809e720 msgr2=0x7f80681491d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 --2- 172.21.15.12:0/2893837064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806809e720 0x7f80681491d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f806809eba0 tx=0x7f805c01c4f0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- 172.21.15.12:0/2893837064 shutdown_connections 2024-04-27T07:05:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 --2- 172.21.15.12:0/2893837064 >> v2:172.21.15.53:3300/0 conn(0x7f806809ce40 0x7f8068149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 --2- 172.21.15.12:0/2893837064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806809e720 0x7f80681491d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 --2- 172.21.15.12:0/2893837064 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f806809ca60 0x7f806806da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- 172.21.15.12:0/2893837064 >> 172.21.15.12:0/2893837064 conn(0x7f80680b9840 msgr2=0x7f80680b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:30.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- 172.21.15.12:0/2893837064 shutdown_connections 2024-04-27T07:05:30.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- 172.21.15.12:0/2893837064 wait complete. 2024-04-27T07:05:30.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 Processor -- start 2024-04-27T07:05:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- start start 2024-04-27T07:05:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806809ca60 0x7f806813dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806809ce40 0x7f8068142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f806809e720 0x7f80680afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f80681459c0 con 0x7f806809ca60 2024-04-27T07:05:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f806814a060 con 0x7f806809ce40 2024-04-27T07:05:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8068145b40 con 0x7f806809e720 2024-04-27T07:05:30.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806cfd0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806809ca60 0x7f806813dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806cfd0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806809ca60 0x7f806813dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55738/0 (socket says 172.21.15.12:55738) 2024-04-27T07:05:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806cfd0640 1 -- 172.21.15.12:0/1840034915 learned_addr learned my addr 172.21.15.12:0/1840034915 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f8067fff640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806809ce40 0x7f8068142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806d7d1640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f806809e720 0x7f80680afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806cfd0640 1 -- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f806809e720 msgr2=0x7f80680afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806cfd0640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f806809e720 0x7f80680afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806cfd0640 1 -- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806809ce40 msgr2=0x7f8068142fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806cfd0640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f806809ce40 0x7f8068142fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806cfd0640 1 -- 172.21.15.12:0/1840034915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806813d350 con 0x7f806809ca60 2024-04-27T07:05:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806cfd0640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806809ca60 0x7f806813dfa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f805800d800 tx=0x7f805800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f80657fa640 1 -- 172.21.15.12:0/1840034915 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805800b6c0 con 0x7f806809ca60 2024-04-27T07:05:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f80657fa640 1 -- 172.21.15.12:0/1840034915 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8058019070 con 0x7f806809ca60 2024-04-27T07:05:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80680aeda0 con 0x7f806809ca60 2024-04-27T07:05:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f80657fa640 1 -- 172.21.15.12:0/1840034915 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805800ba60 con 0x7f806809ca60 2024-04-27T07:05:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.608+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80680ae570 con 0x7f806809ca60 2024-04-27T07:05:30.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80680af6d0 con 0x7f806809ca60 2024-04-27T07:05:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f80657fa640 1 -- 172.21.15.12:0/1840034915 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f805800bc00 con 0x7f806809ca60 2024-04-27T07:05:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f80657fa640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8054068380 0x7f805406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f80657fa640 1 -- 172.21.15.12:0/1840034915 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f80580875d0 con 0x7f806809ca60 2024-04-27T07:05:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f80657fa640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] conn(0x7f805406e240 0x7f8054070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f80657fa640 1 -- 172.21.15.12:0/1840034915 --> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8058001d40 con 0x7f805406e240 2024-04-27T07:05:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f80657fa640 1 -- 172.21.15.12:0/1840034915 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8058087980 con 0x7f806809ca60 2024-04-27T07:05:30.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f8067fff640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8054068380 0x7f805406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f806d7d1640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] conn(0x7f805406e240 0x7f8054070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f8067fff640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8054068380 0x7f805406a840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f805c0ae000 tx=0x7f805c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.612+0000 7f806d7d1640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] conn(0x7f805406e240 0x7f8054070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.616+0000 7f80657fa640 1 -- 172.21.15.12:0/1840034915 <== osd.6 v2:172.21.15.53:6808/289427539 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8058001d40 con 0x7f805406e240 2024-04-27T07:05:30.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 --> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8068068d00 con 0x7f805406e240 2024-04-27T07:05:30.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f80657fa640 1 -- 172.21.15.12:0/1840034915 <== osd.6 v2:172.21.15.53:6808/289427539 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8068068d00 con 0x7f805406e240 2024-04-27T07:05:30.635 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:05:30.635 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:05:30.635 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:05:30.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] conn(0x7f805406e240 msgr2=0x7f8054070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:6808/289427539,v1:172.21.15.53:6809/289427539] conn(0x7f805406e240 0x7f8054070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8054068380 msgr2=0x7f805406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8054068380 0x7f805406a840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f805c0ae000 tx=0x7f805c008040 comp rx=0 tx=0).stop 2024-04-27T07:05:30.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806809ca60 msgr2=0x7f806813dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 --2- 172.21.15.12:0/1840034915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f806809ca60 0x7f806813dfa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f805800d800 tx=0x7f805800dcc0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806d7d1640 1 -- 172.21.15.12:0/1840034915 reap_dead start 2024-04-27T07:05:30.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 shutdown_connections 2024-04-27T07:05:30.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 >> 172.21.15.12:0/1840034915 conn(0x7f80680b9840 msgr2=0x7f80680b6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:30.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 shutdown_connections 2024-04-27T07:05:30.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.628+0000 7f806f234640 1 -- 172.21.15.12:0/1840034915 wait complete. 2024-04-27T07:05:30.652 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-27T07:05:30.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e4322a640 1 Processor -- start 2024-04-27T07:05:30.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e4322a640 1 -- start start 2024-04-27T07:05:30.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e4322a640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e3c1489d0 0x7f3e3c0b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e4322a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3c0b79b0 con 0x7f3e3c1419d0 2024-04-27T07:05:30.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e4322a640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3c0b69f0 con 0x7f3e3c0b6610 2024-04-27T07:05:30.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e4322a640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3c147ef0 con 0x7f3e3c1489d0 2024-04-27T07:05:30.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e417c7640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3e3c1419d0 0x7f3e3c141e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52274/0 (socket says 172.21.15.12:52274) 2024-04-27T07:05:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e417c7640 1 -- 172.21.15.12:0/1563373365 learned_addr learned my addr 172.21.15.12:0/1563373365 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3bfff640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e3c1489d0 0x7f3e3c0b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470871972 0 0) 0x7f3e3c0b79b0 con 0x7f3e3c1419d0 2024-04-27T07:05:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e24003530 con 0x7f3e3c1419d0 2024-04-27T07:05:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2852528824 0 0) 0x7f3e24003530 con 0x7f3e3c1419d0 2024-04-27T07:05:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e3c1489d0 msgr2=0x7f3e3c0b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e3c1489d0 0x7f3e3c0b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 >> v1:172.21.15.53:6789/0 conn(0x7f3e3c0b6610 legacy=0x7f3e3c0b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c1481d0 con 0x7f3e3c1419d0 2024-04-27T07:05:30.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f3e34001a00 con 0x7f3e3c1419d0 2024-04-27T07:05:30.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 >> v1:172.21.15.12:6789/0 conn(0x7f3e3c1419d0 legacy=0x7f3e3c141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e240039f0 0x7f3e24003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e3c1489d0 0x7f3e24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e24004d50 0x7f3e240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c147ef0 con 0x7f3e24004d50 2024-04-27T07:05:30.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2400bb30 con 0x7f3e240039f0 2024-04-27T07:05:30.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2400bd30 con 0x7f3e3c1489d0 2024-04-27T07:05:30.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.768+0000 7f3e417c7640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e24004d50 0x7f3e240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e40fc6640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e240039f0 0x7f3e24003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e3bfff640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e3c1489d0 0x7f3e24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e417c7640 1 -- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e3c1489d0 msgr2=0x7f3e24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:30.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e417c7640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e3c1489d0 0x7f3e24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e417c7640 1 -- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e240039f0 msgr2=0x7f3e24003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:30.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e417c7640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e240039f0 0x7f3e24003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e417c7640 1 -- 172.21.15.12:0/1563373365 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e24003530 con 0x7f3e24004d50 2024-04-27T07:05:30.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e417c7640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e24004d50 0x7f3e240051d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400a860 tx=0x7f3e3401fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b810a42fbb0c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400a480 con 0x7f3e24004d50 2024-04-27T07:05:30.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e340aa710 con 0x7f3e24004d50 2024-04-27T07:05:30.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e3b7fe640 1 -- 172.21.15.12:0/1563373365 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e340aab20 con 0x7f3e24004d50 2024-04-27T07:05:30.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e4322a640 1 -- 172.21.15.12:0/1563373365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e24004d50 msgr2=0x7f3e240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e4322a640 1 --2- 172.21.15.12:0/1563373365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e24004d50 0x7f3e240051d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400a860 tx=0x7f3e3401fae0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e417c7640 1 -- 172.21.15.12:0/1563373365 reap_dead start 2024-04-27T07:05:30.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e4322a640 1 -- 172.21.15.12:0/1563373365 shutdown_connections 2024-04-27T07:05:30.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e4322a640 1 -- 172.21.15.12:0/1563373365 >> 172.21.15.12:0/1563373365 conn(0x7f3e3c0ba140 msgr2=0x7f3e3c0ba550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:30.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e4322a640 1 -- 172.21.15.12:0/1563373365 shutdown_connections 2024-04-27T07:05:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.772+0000 7f3e4322a640 1 -- 172.21.15.12:0/1563373365 wait complete. 2024-04-27T07:05:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 Processor -- start 2024-04-27T07:05:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 -- start start 2024-04-27T07:05:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e24004d50 0x7f3e3c13a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e3c1de630 0x7f3e3c142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e3c1e0f20 0x7f3e3c1e13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c0b6040 con 0x7f3e3c1de630 2024-04-27T07:05:30.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c0b5ec0 con 0x7f3e3c1e0f20 2024-04-27T07:05:30.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3c0b8020 con 0x7f3e24004d50 2024-04-27T07:05:30.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e3c1de630 0x7f3e3c142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e3c1de630 0x7f3e3c142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55754/0 (socket says 172.21.15.12:55754) 2024-04-27T07:05:30.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e3bfff640 1 -- 172.21.15.12:0/1274496705 learned_addr learned my addr 172.21.15.12:0/1274496705 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:30.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e417c7640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e3c1e0f20 0x7f3e3c1e13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e40fc6640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e24004d50 0x7f3e3c13a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e3bfff640 1 -- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e24004d50 msgr2=0x7f3e3c13a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e3bfff640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e24004d50 0x7f3e3c13a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e3bfff640 1 -- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e3c1e0f20 msgr2=0x7f3e3c1e13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e3bfff640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e3c1e0f20 0x7f3e3c1e13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e3bfff640 1 -- 172.21.15.12:0/1274496705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c143390 con 0x7f3e3c1de630 2024-04-27T07:05:30.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e3bfff640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e3c1de630 0x7f3e3c142e50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3e30011620 tx=0x7f3e30011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e397fa640 1 -- 172.21.15.12:0/1274496705 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000f920 con 0x7f3e3c1de630 2024-04-27T07:05:30.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e397fa640 1 -- 172.21.15.12:0/1274496705 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3000fac0 con 0x7f3e3c1de630 2024-04-27T07:05:30.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 -- 172.21.15.12:0/1274496705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c0b8440 con 0x7f3e3c1de630 2024-04-27T07:05:30.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e397fa640 1 -- 172.21.15.12:0/1274496705 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30015460 con 0x7f3e3c1de630 2024-04-27T07:05:30.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 -- 172.21.15.12:0/1274496705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c1dde30 con 0x7f3e3c1de630 2024-04-27T07:05:30.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e4322a640 1 -- 172.21.15.12:0/1274496705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3c069330 con 0x7f3e3c1de630 2024-04-27T07:05:30.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.776+0000 7f3e397fa640 1 -- 172.21.15.12:0/1274496705 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3e30015600 con 0x7f3e3c1de630 2024-04-27T07:05:30.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.780+0000 7f3e397fa640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3e28068380 0x7f3e2806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:30.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.780+0000 7f3e397fa640 1 -- 172.21.15.12:0/1274496705 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f3e30086fc0 con 0x7f3e3c1de630 2024-04-27T07:05:30.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.780+0000 7f3e40fc6640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3e28068380 0x7f3e2806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:30.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.780+0000 7f3e40fc6640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3e28068380 0x7f3e2806a840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c007fd0 tx=0x7f3e2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:30.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.784+0000 7f3e397fa640 1 -- 172.21.15.12:0/1274496705 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3e30019030 con 0x7f3e3c1de630 2024-04-27T07:05:30.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.940+0000 7f3e4322a640 1 -- 172.21.15.12:0/1274496705 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f3e3c145770 con 0x7f3e28068380 2024-04-27T07:05:30.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.940+0000 7f3e397fa640 1 -- 172.21.15.12:0/1274496705 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3e3c145770 con 0x7f3e28068380 2024-04-27T07:05:30.948 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 6 to deep-scrub 2024-04-27T07:05:30.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 -- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3e28068380 msgr2=0x7f3e2806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3e28068380 0x7f3e2806a840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c007fd0 tx=0x7f3e2c009040 comp rx=0 tx=0).stop 2024-04-27T07:05:30.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 -- 172.21.15.12:0/1274496705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e3c1de630 msgr2=0x7f3e3c142e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:30.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e3c1de630 0x7f3e3c142e50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3e30011620 tx=0x7f3e30011ae0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 -- 172.21.15.12:0/1274496705 shutdown_connections 2024-04-27T07:05:30.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3e28068380 0x7f3e2806a840 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e3c1e0f20 0x7f3e3c1e13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e3c1de630 0x7f3e3c142e50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 --2- 172.21.15.12:0/1274496705 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3e24004d50 0x7f3e3c13a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:30.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 -- 172.21.15.12:0/1274496705 >> 172.21.15.12:0/1274496705 conn(0x7f3e3c0ba140 msgr2=0x7f3e3c06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:30.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 -- 172.21.15.12:0/1274496705 shutdown_connections 2024-04-27T07:05:30.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:30.944+0000 7f3e4322a640 1 -- 172.21.15.12:0/1274496705 wait complete. 2024-04-27T07:05:30.965 INFO:tasks.ceph:Scrubbing osd.7 2024-04-27T07:05:30.966 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-27T07:05:31.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed20c97640 1 Processor -- start 2024-04-27T07:05:31.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed20c97640 1 -- start start 2024-04-27T07:05:31.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed20c97640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fed1c13fe10 0x7fed1c0b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed20c97640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fed1c1401f0 0x7fed1c1465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed20c97640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c0b8dc0 0x7fed1c149750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed20c97640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fed1c14a190 con 0x7fed1c1401f0 2024-04-27T07:05:31.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed20c97640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fed1c145db0 con 0x7fed1c13fe10 2024-04-27T07:05:31.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed20c97640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fed1c146090 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed19d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fed1c1401f0 0x7fed1c1465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed19d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fed1c1401f0 0x7fed1c1465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55760/0 (socket says 172.21.15.12:55760) 2024-04-27T07:05:31.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed19d9b640 1 -- 172.21.15.12:0/1502865617 learned_addr learned my addr 172.21.15.12:0/1502865617 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed1ad9d640 1 --2- 172.21.15.12:0/1502865617 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c0b8dc0 0x7fed1c149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed1a59c640 1 --2- 172.21.15.12:0/1502865617 >> v2:172.21.15.53:3300/0 conn(0x7fed1c13fe10 0x7fed1c0b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed19d9b640 1 -- 172.21.15.12:0/1502865617 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c0b8dc0 msgr2=0x7fed1c149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed19d9b640 1 --2- 172.21.15.12:0/1502865617 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c0b8dc0 0x7fed1c149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed19d9b640 1 -- 172.21.15.12:0/1502865617 >> v2:172.21.15.53:3300/0 conn(0x7fed1c13fe10 msgr2=0x7fed1c0b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed19d9b640 1 --2- 172.21.15.12:0/1502865617 >> v2:172.21.15.53:3300/0 conn(0x7fed1c13fe10 0x7fed1c0b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed19d9b640 1 -- 172.21.15.12:0/1502865617 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c06e5c0 con 0x7fed1c1401f0 2024-04-27T07:05:31.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.084+0000 7fed1ad9d640 1 --2- 172.21.15.12:0/1502865617 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c0b8dc0 0x7fed1c149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:31.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed19d9b640 1 --2- 172.21.15.12:0/1502865617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed1c1401f0 0x7fed1c1465b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0ba630 tx=0x7fed1001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a3daa31a3ffa8f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:31.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed1959a640 1 -- 172.21.15.12:0/1502865617 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed100b2070 con 0x7fed1c1401f0 2024-04-27T07:05:31.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed1959a640 1 -- 172.21.15.12:0/1502865617 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed100a9070 con 0x7fed1c1401f0 2024-04-27T07:05:31.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed1959a640 1 -- 172.21.15.12:0/1502865617 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed100ad880 con 0x7fed1c1401f0 2024-04-27T07:05:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 -- 172.21.15.12:0/1502865617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed1c1401f0 msgr2=0x7fed1c1465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 --2- 172.21.15.12:0/1502865617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed1c1401f0 0x7fed1c1465b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0ba630 tx=0x7fed1001c490 comp rx=0 tx=0).stop 2024-04-27T07:05:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 -- 172.21.15.12:0/1502865617 shutdown_connections 2024-04-27T07:05:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 --2- 172.21.15.12:0/1502865617 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c0b8dc0 0x7fed1c149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 --2- 172.21.15.12:0/1502865617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed1c1401f0 0x7fed1c1465b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 --2- 172.21.15.12:0/1502865617 >> v2:172.21.15.53:3300/0 conn(0x7fed1c13fe10 0x7fed1c0b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 -- 172.21.15.12:0/1502865617 >> 172.21.15.12:0/1502865617 conn(0x7fed1c09c480 msgr2=0x7fed1c09c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:31.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 -- 172.21.15.12:0/1502865617 shutdown_connections 2024-04-27T07:05:31.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 -- 172.21.15.12:0/1502865617 wait complete. 2024-04-27T07:05:31.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 Processor -- start 2024-04-27T07:05:31.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.088+0000 7fed20c97640 1 -- start start 2024-04-27T07:05:31.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed20c97640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed1c0b8dc0 0x7fed1c139d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed20c97640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c13fe10 0x7fed1c1debb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed20c97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed1c1401f0 0x7fed1c1df0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed20c97640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c06c760 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed20c97640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c06c5e0 con 0x7fed1c1401f0 2024-04-27T07:05:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed20c97640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fed1c06c8e0 con 0x7fed1c13fe10 2024-04-27T07:05:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed1c0b8dc0 0x7fed1c139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed1c0b8dc0 0x7fed1c139d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55762/0 (socket says 172.21.15.12:55762) 2024-04-27T07:05:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1a59c640 1 -- 172.21.15.12:0/2513947225 learned_addr learned my addr 172.21.15.12:0/2513947225 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed19d9b640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c13fe10 0x7fed1c1debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1ad9d640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed1c1401f0 0x7fed1c1df0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1a59c640 1 -- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c13fe10 msgr2=0x7fed1c1debb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1a59c640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c13fe10 0x7fed1c1debb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1a59c640 1 -- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed1c1401f0 msgr2=0x7fed1c1df0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1a59c640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed1c1401f0 0x7fed1c1df0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1a59c640 1 -- 172.21.15.12:0/2513947225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c139590 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed19d9b640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fed1c13fe10 0x7fed1c1debb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1ad9d640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed1c1401f0 0x7fed1c1df0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1a59c640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed1c0b8dc0 0x7fed1c139d70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00d800 tx=0x7fed0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:31.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed0affd640 1 -- 172.21.15.12:0/2513947225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00b6c0 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed0affd640 1 -- 172.21.15.12:0/2513947225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0c019070 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1c139870 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed0affd640 1 -- 172.21.15.12:0/2513947225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00ba60 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1c13a6c0 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed1c13b770 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed0affd640 1 -- 172.21.15.12:0/2513947225 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fed0c00bc00 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed0affd640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fecfc068380 0x7fecfc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed0affd640 1 -- 172.21.15.12:0/2513947225 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fed0c0875d0 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed0affd640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] conn(0x7fecfc06e240 0x7fecfc070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed0affd640 1 -- 172.21.15.12:0/2513947225 --> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed0c001d40 con 0x7fecfc06e240 2024-04-27T07:05:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed0affd640 1 -- 172.21.15.12:0/2513947225 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed0c087a60 con 0x7fed1c0b8dc0 2024-04-27T07:05:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed19d9b640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fecfc068380 0x7fecfc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.092+0000 7fed1ad9d640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] conn(0x7fecfc06e240 0x7fecfc070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.096+0000 7fed19d9b640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fecfc068380 0x7fecfc06a840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fed100af000 tx=0x7fed10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:31.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.096+0000 7fed1ad9d640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] conn(0x7fecfc06e240 0x7fecfc070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:31.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.096+0000 7fed0affd640 1 -- 172.21.15.12:0/2513947225 <== osd.7 v2:172.21.15.53:6816/2338388162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed0c001d40 con 0x7fecfc06e240 2024-04-27T07:05:31.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.108+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 --> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fed1c069330 con 0x7fecfc06e240 2024-04-27T07:05:31.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.108+0000 7fed0affd640 1 -- 172.21.15.12:0/2513947225 <== osd.7 v2:172.21.15.53:6816/2338388162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fed1c069330 con 0x7fecfc06e240 2024-04-27T07:05:31.116 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-27T07:05:31.116 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T07:05:31.116 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-27T07:05:31.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.108+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] conn(0x7fecfc06e240 msgr2=0x7fecfc070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.108+0000 7fed20c97640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:6816/2338388162,v1:172.21.15.53:6817/2338388162] conn(0x7fecfc06e240 0x7fecfc070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.112+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fecfc068380 msgr2=0x7fecfc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.112+0000 7fed20c97640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fecfc068380 0x7fecfc06a840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fed100af000 tx=0x7fed10008040 comp rx=0 tx=0).stop 2024-04-27T07:05:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.112+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed1c0b8dc0 msgr2=0x7fed1c139d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.112+0000 7fed20c97640 1 --2- 172.21.15.12:0/2513947225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed1c0b8dc0 0x7fed1c139d70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00d800 tx=0x7fed0c00dcc0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.112+0000 7fed1ad9d640 1 -- 172.21.15.12:0/2513947225 reap_dead start 2024-04-27T07:05:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.112+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 shutdown_connections 2024-04-27T07:05:31.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.112+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 >> 172.21.15.12:0/2513947225 conn(0x7fed1c09c480 msgr2=0x7fed1c06e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.112+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 shutdown_connections 2024-04-27T07:05:31.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.112+0000 7fed20c97640 1 -- 172.21.15.12:0/2513947225 wait complete. 2024-04-27T07:05:31.131 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-27T07:05:31.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf0e50d640 1 Processor -- start 2024-04-27T07:05:31.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf0e50d640 1 -- start start 2024-04-27T07:05:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf0e50d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbf080b5f30 0x7fbf08141980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf0e50d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf08141ec0 0x7fbf081525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf0e50d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf080b4dd0 con 0x7fbf080bdf70 2024-04-27T07:05:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf0e50d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7fbf080b63c0 con 0x7fbf080b5f30 2024-04-27T07:05:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf0e50d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fbf080b50d0 con 0x7fbf08141ec0 2024-04-27T07:05:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf077fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbf080b5f30 0x7fbf08141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf07fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbf080bdf70 0x7fbf0814b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52290/0 (socket says 172.21.15.12:52290) 2024-04-27T07:05:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf07fff640 1 -- 172.21.15.12:0/3526087767 learned_addr learned my addr 172.21.15.12:0/3526087767 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf0caaa640 1 --2- 172.21.15.12:0/3526087767 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf08141ec0 0x7fbf081525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf06ffd640 1 -- 172.21.15.12:0/3526087767 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 84833936 0 0) 0x7fbf080b4dd0 con 0x7fbf080bdf70 2024-04-27T07:05:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf077fe640 1 -- 172.21.15.12:0/3526087767 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf08141ec0 msgr2=0x7fbf081525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:31.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf077fe640 1 --2- 172.21.15.12:0/3526087767 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf08141ec0 0x7fbf081525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf077fe640 1 -- 172.21.15.12:0/3526087767 >> v1:172.21.15.12:6789/0 conn(0x7fbf080bdf70 legacy=0x7fbf0814b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf077fe640 1 -- 172.21.15.12:0/3526087767 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf080b5560 con 0x7fbf080b5f30 2024-04-27T07:05:31.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.240+0000 7fbf0caaa640 1 --2- 172.21.15.12:0/3526087767 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf08141ec0 0x7fbf081525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:31.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf077fe640 1 --2- 172.21.15.12:0/3526087767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 0x7fbf08141980 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009d90 tx=0x7fbef001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab70fe0a60b7991 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:31.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf06ffd640 1 -- 172.21.15.12:0/3526087767 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00b2070 con 0x7fbf080b5f30 2024-04-27T07:05:31.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf06ffd640 1 -- 172.21.15.12:0/3526087767 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef00a57a0 con 0x7fbf080b5f30 2024-04-27T07:05:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf06ffd640 1 -- 172.21.15.12:0/3526087767 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00a5b70 con 0x7fbf080b5f30 2024-04-27T07:05:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 -- 172.21.15.12:0/3526087767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 msgr2=0x7fbf08141980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 --2- 172.21.15.12:0/3526087767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 0x7fbf08141980 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009d90 tx=0x7fbef001c8e0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 -- 172.21.15.12:0/3526087767 shutdown_connections 2024-04-27T07:05:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 --2- 172.21.15.12:0/3526087767 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf08141ec0 0x7fbf081525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 --2- 172.21.15.12:0/3526087767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 0x7fbf08141980 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 -- 172.21.15.12:0/3526087767 >> 172.21.15.12:0/3526087767 conn(0x7fbf080738b0 msgr2=0x7fbf080bd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:31.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 -- 172.21.15.12:0/3526087767 shutdown_connections 2024-04-27T07:05:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 -- 172.21.15.12:0/3526087767 wait complete. 2024-04-27T07:05:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 Processor -- start 2024-04-27T07:05:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 -- start start 2024-04-27T07:05:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 0x7fbf0813bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf080bdf70 0x7fbf08145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf08141ec0 0x7fbf081e3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbf080b6780 con 0x7fbf080b5f30 2024-04-27T07:05:31.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbf080b6600 con 0x7fbf08141ec0 2024-04-27T07:05:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0e50d640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fbf08155340 con 0x7fbf080bdf70 2024-04-27T07:05:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf07fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 0x7fbf0813bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf07fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 0x7fbf0813bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55776/0 (socket says 172.21.15.12:55776) 2024-04-27T07:05:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf07fff640 1 -- 172.21.15.12:0/2539002344 learned_addr learned my addr 172.21.15.12:0/2539002344 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf077fe640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf080bdf70 0x7fbf08145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf0caaa640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf08141ec0 0x7fbf081e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf07fff640 1 -- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf080bdf70 msgr2=0x7fbf08145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf07fff640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf080bdf70 0x7fbf08145ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf07fff640 1 -- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf08141ec0 msgr2=0x7fbf081e3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf07fff640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf08141ec0 0x7fbf081e3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.244+0000 7fbf07fff640 1 -- 172.21.15.12:0/2539002344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0813de40 con 0x7fbf080b5f30 2024-04-27T07:05:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf07fff640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 0x7fbf0813bb90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbf000011f0 tx=0x7fbf0000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf04ff9640 1 -- 172.21.15.12:0/2539002344 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00018070 con 0x7fbf080b5f30 2024-04-27T07:05:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf04ff9640 1 -- 172.21.15.12:0/2539002344 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf000074e0 con 0x7fbf080b5f30 2024-04-27T07:05:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf04ff9640 1 -- 172.21.15.12:0/2539002344 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00004a10 con 0x7fbf080b5f30 2024-04-27T07:05:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf0e50d640 1 -- 172.21.15.12:0/2539002344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf081478a0 con 0x7fbf080b5f30 2024-04-27T07:05:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf0e50d640 1 -- 172.21.15.12:0/2539002344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0813e0e0 con 0x7fbf080b5f30 2024-04-27T07:05:31.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf0e50d640 1 -- 172.21.15.12:0/2539002344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf08068320 con 0x7fbf080b5f30 2024-04-27T07:05:31.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf04ff9640 1 -- 172.21.15.12:0/2539002344 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbf00004bb0 con 0x7fbf080b5f30 2024-04-27T07:05:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf04ff9640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbee8068380 0x7fbee806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf04ff9640 1 -- 172.21.15.12:0/2539002344 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fbf000859e0 con 0x7fbf080b5f30 2024-04-27T07:05:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf077fe640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbee8068380 0x7fbee806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.248+0000 7fbf077fe640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbee8068380 0x7fbee806a840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbef00af000 tx=0x7fbef00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:31.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.252+0000 7fbf04ff9640 1 -- 172.21.15.12:0/2539002344 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbf00056330 con 0x7fbf080b5f30 2024-04-27T07:05:31.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.416+0000 7fbf0e50d640 1 -- 172.21.15.12:0/2539002344 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fbf080b9d80 con 0x7fbee8068380 2024-04-27T07:05:31.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.416+0000 7fbf04ff9640 1 -- 172.21.15.12:0/2539002344 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbf080b9d80 con 0x7fbee8068380 2024-04-27T07:05:31.423 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 7 to deep-scrub 2024-04-27T07:05:31.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 -- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbee8068380 msgr2=0x7fbee806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbee8068380 0x7fbee806a840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbef00af000 tx=0x7fbef00a7070 comp rx=0 tx=0).stop 2024-04-27T07:05:31.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 -- 172.21.15.12:0/2539002344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 msgr2=0x7fbf0813bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 0x7fbf0813bb90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbf000011f0 tx=0x7fbf0000c9b0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 -- 172.21.15.12:0/2539002344 shutdown_connections 2024-04-27T07:05:31.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fbee8068380 0x7fbee806a840 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf08141ec0 0x7fbf081e3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fbf080bdf70 0x7fbf08145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 --2- 172.21.15.12:0/2539002344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf080b5f30 0x7fbf0813bb90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 -- 172.21.15.12:0/2539002344 >> 172.21.15.12:0/2539002344 conn(0x7fbf080738b0 msgr2=0x7fbf080b86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:31.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 -- 172.21.15.12:0/2539002344 shutdown_connections 2024-04-27T07:05:31.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.420+0000 7fbf0e50d640 1 -- 172.21.15.12:0/2539002344 wait complete. 2024-04-27T07:05:31.441 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:05:31.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.556+0000 7f8fdce04640 1 Processor -- start 2024-04-27T07:05:31.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.556+0000 7f8fdce04640 1 -- start start 2024-04-27T07:05:31.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fdce04640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8fd80b9a10 0x7f8fd80b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fdce04640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fd806dac0 con 0x7f8fd814e400 2024-04-27T07:05:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fdce04640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fd806e890 con 0x7f8fd80b3680 2024-04-27T07:05:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fdce04640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f8fd80b9df0 con 0x7f8fd80b9a10 2024-04-27T07:05:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8fd80b9a10 0x7f8fd80b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8fd80b9a10 0x7f8fd80b65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55780/0 (socket says 172.21.15.12:55780) 2024-04-27T07:05:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd659c640 1 -- 172.21.15.12:0/1652376677 learned_addr learned my addr 172.21.15.12:0/1652376677 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd6d9d640 1 --1- 172.21.15.12:0/1652376677 >> v1:172.21.15.12:6789/0 conn(0x7f8fd814e400 0x7f8fd81507f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52292/0 (socket says 172.21.15.12:52292) 2024-04-27T07:05:31.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd659c640 1 -- 172.21.15.12:0/1652376677 >> v1:172.21.15.53:6789/0 conn(0x7f8fd80b3680 legacy=0x7f8fd8145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd659c640 1 -- 172.21.15.12:0/1652376677 >> v1:172.21.15.12:6789/0 conn(0x7f8fd814e400 legacy=0x7f8fd81507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd559a640 1 -- 172.21.15.12:0/1652376677 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1195167178 0 0) 0x7f8fc8001120 con 0x7f8fd814e400 2024-04-27T07:05:31.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd659c640 1 -- 172.21.15.12:0/1652376677 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd806e090 con 0x7f8fd80b9a10 2024-04-27T07:05:31.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd659c640 1 --2- 172.21.15.12:0/1652376677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd80b9a10 0x7f8fd80b65c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009870 tx=0x7f8fd001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e51a6380baa5d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:31.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd559a640 1 -- 172.21.15.12:0/1652376677 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd00b0070 con 0x7f8fd80b9a10 2024-04-27T07:05:31.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd559a640 1 -- 172.21.15.12:0/1652376677 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd00ab600 con 0x7f8fd80b9a10 2024-04-27T07:05:31.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.560+0000 7f8fd559a640 1 -- 172.21.15.12:0/1652376677 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd00ab9d0 con 0x7f8fd80b9a10 2024-04-27T07:05:31.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- 172.21.15.12:0/1652376677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd80b9a10 msgr2=0x7f8fd80b65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 --2- 172.21.15.12:0/1652376677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd80b9a10 0x7f8fd80b65c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009870 tx=0x7f8fd001c4d0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- 172.21.15.12:0/1652376677 shutdown_connections 2024-04-27T07:05:31.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 --2- 172.21.15.12:0/1652376677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd80b9a10 0x7f8fd80b65c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- 172.21.15.12:0/1652376677 >> 172.21.15.12:0/1652376677 conn(0x7f8fd8067d10 msgr2=0x7f8fd8068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:31.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- 172.21.15.12:0/1652376677 shutdown_connections 2024-04-27T07:05:31.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- 172.21.15.12:0/1652376677 wait complete. 2024-04-27T07:05:31.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 Processor -- start 2024-04-27T07:05:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- start start 2024-04-27T07:05:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8fd80b3680 0x7f8fd8150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80b9a10 0x7f8fd809c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd814e400 0x7f8fd809ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8074020 con 0x7f8fd814e400 2024-04-27T07:05:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8073ea0 con 0x7f8fd80b9a10 2024-04-27T07:05:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8fd80b6b00 con 0x7f8fd80b3680 2024-04-27T07:05:31.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd814e400 0x7f8fd809ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd814e400 0x7f8fd809ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55790/0 (socket says 172.21.15.12:55790) 2024-04-27T07:05:31.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd6d9d640 1 -- 172.21.15.12:0/167640127 learned_addr learned my addr 172.21.15.12:0/167640127 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:31.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd659c640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8fd80b3680 0x7f8fd8150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd5d9b640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80b9a10 0x7f8fd809c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd6d9d640 1 -- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8fd80b3680 msgr2=0x7f8fd8150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd6d9d640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8fd80b3680 0x7f8fd8150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd6d9d640 1 -- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80b9a10 msgr2=0x7f8fd809c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd6d9d640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80b9a10 0x7f8fd809c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd6d9d640 1 -- 172.21.15.12:0/167640127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd8150460 con 0x7f8fd814e400 2024-04-27T07:05:31.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd659c640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8fd80b3680 0x7f8fd8150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:31.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd5d9b640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80b9a10 0x7f8fd809c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:31.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fd6d9d640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd814e400 0x7f8fd809ced0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800c440 tx=0x7f8fc800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:31.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fceffd640 1 -- 172.21.15.12:0/167640127 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc80074e0 con 0x7f8fd814e400 2024-04-27T07:05:31.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fceffd640 1 -- 172.21.15.12:0/167640127 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc80047f0 con 0x7f8fd814e400 2024-04-27T07:05:31.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- 172.21.15.12:0/167640127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd813d9c0 con 0x7f8fd814e400 2024-04-27T07:05:31.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fceffd640 1 -- 172.21.15.12:0/167640127 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8004be0 con 0x7f8fd814e400 2024-04-27T07:05:31.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- 172.21.15.12:0/167640127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd8149e90 con 0x7f8fd814e400 2024-04-27T07:05:31.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.564+0000 7f8fdce04640 1 -- 172.21.15.12:0/167640127 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd8068c20 con 0x7f8fd814e400 2024-04-27T07:05:31.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.568+0000 7f8fceffd640 1 -- 172.21.15.12:0/167640127 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8fc8011070 con 0x7f8fd814e400 2024-04-27T07:05:31.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.568+0000 7f8fceffd640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8fb8068380 0x7f8fb806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:31.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.568+0000 7f8fceffd640 1 -- 172.21.15.12:0/167640127 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f8fc80861f0 con 0x7f8fd814e400 2024-04-27T07:05:31.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.568+0000 7f8fd659c640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8fb8068380 0x7f8fb806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:31.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.568+0000 7f8fd659c640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8fb8068380 0x7f8fb806a840 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00ad000 tx=0x7f8fd00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:31.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.572+0000 7f8fceffd640 1 -- 172.21.15.12:0/167640127 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8fd806dac0 con 0x7f8fd814e400 2024-04-27T07:05:31.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.732+0000 7f8fdce04640 1 -- 172.21.15.12:0/167640127 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fd80b2cc0 con 0x7f8fb8068380 2024-04-27T07:05:31.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.732+0000 7f8fceffd640 1 -- 172.21.15.12:0/167640127 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13348 (secure 0 0 0) 0x7f8fd80b2cc0 con 0x7f8fb8068380 2024-04-27T07:05:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:31.739 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:05:31.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 -- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8fb8068380 msgr2=0x7f8fb806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8fb8068380 0x7f8fb806a840 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00ad000 tx=0x7f8fd00a6000 comp rx=0 tx=0).stop 2024-04-27T07:05:31.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 -- 172.21.15.12:0/167640127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd814e400 msgr2=0x7f8fd809ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd814e400 0x7f8fd809ced0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800c440 tx=0x7f8fc800c840 comp rx=0 tx=0).stop 2024-04-27T07:05:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 -- 172.21.15.12:0/167640127 shutdown_connections 2024-04-27T07:05:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8fb8068380 0x7f8fb806a840 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd814e400 0x7f8fd809ced0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd80b9a10 0x7f8fd809c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 --2- 172.21.15.12:0/167640127 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8fd80b3680 0x7f8fd8150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 -- 172.21.15.12:0/167640127 >> 172.21.15.12:0/167640127 conn(0x7f8fd8067d10 msgr2=0x7f8fd80b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 -- 172.21.15.12:0/167640127 shutdown_connections 2024-04-27T07:05:31.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:31.736+0000 7f8fdce04640 1 -- 172.21.15.12:0/167640127 wait complete. 2024-04-27T07:05:31.757 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-27T07:05:30.633840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236840,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737880,"statfs":{"total":773094113280,"available":772851589120,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.571509"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-27T07:05:19.386254+0000","last_change":"2024-04-27T07:04:10.689537+0000","last_active":"2024-04-27T07:05:19.386254+0000","last_peered":"2024-04-27T07:05:19.386254+0000","last_clean":"2024-04-27T07:05:19.386254+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:05:19.386254+0000","last_undegraded":"2024-04-27T07:05:19.386254+0000","last_fullsized":"2024-04-27T07:05:19.386254+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T07:04:09.534752+0000","last_clean_scrub_stamp":"2024-04-27T07:04:09.534752+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.446418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:05:31.759 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-27T07:04:09.534752+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=7, tm_min=4, tm_sec=9, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=7, tm_min=5, tm_sec=26, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T07:05:31.759 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-27T07:05:51.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T07:05:51.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.888+0000 7f8b4cb30640 1 Processor -- start 2024-04-27T07:05:51.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.888+0000 7f8b4cb30640 1 -- start start 2024-04-27T07:05:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4cb30640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7f8b480b64c0 0x7f8b480b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4cb30640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b480b7560 con 0x7f8b48148810 2024-04-27T07:05:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4cb30640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b480b68a0 con 0x7f8b480b5ae0 2024-04-27T07:05:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4cb30640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7f8b48148c80 con 0x7f8b480b64c0 2024-04-27T07:05:51.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b46d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8b48148810 0x7f8b48152640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44100/0 (socket says 172.21.15.12:44100) 2024-04-27T07:05:51.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b46d9d640 1 -- 172.21.15.12:0/2888917362 learned_addr learned my addr 172.21.15.12:0/2888917362 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:51.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4659c640 1 --2- 172.21.15.12:0/2888917362 >> v2:172.21.15.53:3300/0 conn(0x7f8b480b64c0 0x7f8b480b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:51.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2144700312 0 0) 0x7f8b480b7560 con 0x7f8b48148810 2024-04-27T07:05:51.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b30003530 con 0x7f8b48148810 2024-04-27T07:05:51.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1270349073 0 0) 0x7f8b30003530 con 0x7f8b48148810 2024-04-27T07:05:51.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 >> v2:172.21.15.53:3300/0 conn(0x7f8b480b64c0 msgr2=0x7f8b480b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:51.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 --2- 172.21.15.12:0/2888917362 >> v2:172.21.15.53:3300/0 conn(0x7f8b480b64c0 0x7f8b480b7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:51.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 >> v1:172.21.15.53:6789/0 conn(0x7f8b480b5ae0 legacy=0x7f8b48141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b48148010 con 0x7f8b48148810 2024-04-27T07:05:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3515565878 0 0) 0x7f8b40001a20 con 0x7f8b48148810 2024-04-27T07:05:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 >> v1:172.21.15.12:6789/0 conn(0x7f8b48148810 legacy=0x7f8b48152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8b30004810 0x7f8b30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b300052f0 0x7f8b3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48148c80 con 0x7f8b300039c0 2024-04-27T07:05:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3000c1c0 con 0x7f8b300052f0 2024-04-27T07:05:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3000c3c0 con 0x7f8b30004810 2024-04-27T07:05:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4659c640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b46d9d640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b300052f0 0x7f8b3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b45d9b640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8b30004810 0x7f8b30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4659c640 1 -- 172.21.15.12:0/2888917362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8b30004810 msgr2=0x7f8b30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4659c640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8b30004810 0x7f8b30004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4659c640 1 -- 172.21.15.12:0/2888917362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b300052f0 msgr2=0x7f8b3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4659c640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b300052f0 0x7f8b3000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4659c640 1 -- 172.21.15.12:0/2888917362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b30003530 con 0x7f8b300039c0 2024-04-27T07:05:51.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b46d9d640 1 -- 172.21.15.12:0/2888917362 reap_dead start 2024-04-27T07:05:51.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b45d9b640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8b30004810 0x7f8b30004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T07:05:51.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4659c640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b30003de0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009ab0 tx=0x7f8b3801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c222094d889cb086 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:51.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380096b0 con 0x7f8b300039c0 2024-04-27T07:05:51.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b380a6430 con 0x7f8b300039c0 2024-04-27T07:05:51.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.892+0000 7f8b4559a640 1 -- 172.21.15.12:0/2888917362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380a6820 con 0x7f8b300039c0 2024-04-27T07:05:51.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- 172.21.15.12:0/2888917362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 msgr2=0x7f8b30003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:51.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b30003de0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009ab0 tx=0x7f8b3801ca50 comp rx=0 tx=0).stop 2024-04-27T07:05:51.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- 172.21.15.12:0/2888917362 shutdown_connections 2024-04-27T07:05:51.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 --2- 172.21.15.12:0/2888917362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b30003de0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:51.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- 172.21.15.12:0/2888917362 >> 172.21.15.12:0/2888917362 conn(0x7f8b48073c00 msgr2=0x7f8b480bd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:51.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- 172.21.15.12:0/2888917362 shutdown_connections 2024-04-27T07:05:51.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- 172.21.15.12:0/2888917362 wait complete. 2024-04-27T07:05:51.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 Processor -- start 2024-04-27T07:05:51.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- start start 2024-04-27T07:05:51.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b4813c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:51.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b48142d30 0x7f8b481e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:51.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8b481e4c20 0x7f8b481359a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:51.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4814b2b0 con 0x7f8b300039c0 2024-04-27T07:05:51.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4814b430 con 0x7f8b48142d30 2024-04-27T07:05:51.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4814b5b0 con 0x7f8b481e4c20 2024-04-27T07:05:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b4813c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b4813c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37056/0 (socket says 172.21.15.12:37056) 2024-04-27T07:05:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4659c640 1 -- 172.21.15.12:0/1760073701 learned_addr learned my addr 172.21.15.12:0/1760073701 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4659c640 1 -- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8b481e4c20 msgr2=0x7f8b481359a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4659c640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8b481e4c20 0x7f8b481359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4659c640 1 -- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b48142d30 msgr2=0x7f8b481e46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4659c640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b48142d30 0x7f8b481e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:51.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4659c640 1 -- 172.21.15.12:0/1760073701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b48144a00 con 0x7f8b300039c0 2024-04-27T07:05:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4659c640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b4813c110 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8b3801c880 tx=0x7f8b380197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b36ffd640 1 -- 172.21.15.12:0/1760073701 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380a6ce0 con 0x7f8b300039c0 2024-04-27T07:05:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b36ffd640 1 -- 172.21.15.12:0/1760073701 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b380b9440 con 0x7f8b300039c0 2024-04-27T07:05:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- 172.21.15.12:0/1760073701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b48144ce0 con 0x7f8b300039c0 2024-04-27T07:05:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b36ffd640 1 -- 172.21.15.12:0/1760073701 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380b9810 con 0x7f8b300039c0 2024-04-27T07:05:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.896+0000 7f8b4cb30640 1 -- 172.21.15.12:0/1760073701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b48143190 con 0x7f8b300039c0 2024-04-27T07:05:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.900+0000 7f8b4cb30640 1 -- 172.21.15.12:0/1760073701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b48068c20 con 0x7f8b300039c0 2024-04-27T07:05:51.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.900+0000 7f8b36ffd640 1 -- 172.21.15.12:0/1760073701 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8b380c3410 con 0x7f8b300039c0 2024-04-27T07:05:51.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.900+0000 7f8b36ffd640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b28068380 0x7f8b2806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:51.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.900+0000 7f8b36ffd640 1 -- 172.21.15.12:0/1760073701 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f8b38120790 con 0x7f8b300039c0 2024-04-27T07:05:51.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.900+0000 7f8b45d9b640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b28068380 0x7f8b2806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:51.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.900+0000 7f8b45d9b640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b28068380 0x7f8b2806a840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c007fd0 tx=0x7f8b3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:51.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:51.904+0000 7f8b36ffd640 1 -- 172.21.15.12:0/1760073701 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8b380bf060 con 0x7f8b300039c0 2024-04-27T07:05:52.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.056+0000 7f8b4cb30640 1 -- 172.21.15.12:0/1760073701 --> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b481455e0 con 0x7f8b28068380 2024-04-27T07:05:52.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b36ffd640 1 -- 172.21.15.12:0/1760073701 <== mgr.4105 v2:172.21.15.53:6832/452515146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13354 (secure 0 0 0) 0x7f8b481455e0 con 0x7f8b28068380 2024-04-27T07:05:52.067 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:52.069 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-27T07:05:52.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 -- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b28068380 msgr2=0x7f8b2806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b28068380 0x7f8b2806a840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c007fd0 tx=0x7f8b3c009040 comp rx=0 tx=0).stop 2024-04-27T07:05:52.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 -- 172.21.15.12:0/1760073701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 msgr2=0x7f8b4813c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b4813c110 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8b3801c880 tx=0x7f8b380197f0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 -- 172.21.15.12:0/1760073701 shutdown_connections 2024-04-27T07:05:52.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f8b28068380 0x7f8b2806a840 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f8b481e4c20 0x7f8b481359a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b48142d30 0x7f8b481e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 --2- 172.21.15.12:0/1760073701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b300039c0 0x7f8b4813c110 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 -- 172.21.15.12:0/1760073701 >> 172.21.15.12:0/1760073701 conn(0x7f8b48073c00 msgr2=0x7f8b480b8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:52.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 -- 172.21.15.12:0/1760073701 shutdown_connections 2024-04-27T07:05:52.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.060+0000 7f8b4cb30640 1 -- 172.21.15.12:0/1760073701 wait complete. 2024-04-27T07:05:52.083 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-27T07:05:50.639044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236832,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737888,"statfs":{"total":773094113280,"available":772851597312,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003085"},"pg_stats":[{"pgid":"1.0","version":"32'119","reported_seq":203,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-27T07:05:32.320801+0000","last_change":"2024-04-27T07:05:32.320869+0000","last_active":"2024-04-27T07:05:32.320801+0000","last_peered":"2024-04-27T07:05:32.320801+0000","last_clean":"2024-04-27T07:05:32.320801+0000","last_became_active":"2024-04-27T07:04:10.686510+0000","last_became_peered":"2024-04-27T07:04:10.686510+0000","last_unstale":"2024-04-27T07:05:32.320801+0000","last_undegraded":"2024-04-27T07:05:32.320801+0000","last_fullsized":"2024-04-27T07:05:32.320801+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'119","last_scrub_stamp":"2024-04-27T07:05:32.320621+0000","last_deep_scrub":"32'119","last_deep_scrub_stamp":"2024-04-27T07:05:32.320621+0000","last_clean_scrub_stamp":"2024-04-27T07:05:32.320621+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T18:45:37.774635+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31540,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T07:05:52.084 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-27T07:05:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b091ef640 1 Processor -- start 2024-04-27T07:05:52.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b091ef640 1 -- start start 2024-04-27T07:05:52.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b091ef640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3b040b69a0 0x7f3b04144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b091ef640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3b0409db50 0x7f3b04141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b091ef640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b04144870 con 0x7f3b04141670 2024-04-27T07:05:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b091ef640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 -- 0x7f3b04142010 con 0x7f3b0409db50 2024-04-27T07:05:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b091ef640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3b040b58d0 con 0x7f3b040b69a0 2024-04-27T07:05:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b0259c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3b04141670 0x7f3b0409d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44104/0 (socket says 172.21.15.12:44104) 2024-04-27T07:05:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b0259c640 1 -- 172.21.15.12:0/666612729 learned_addr learned my addr 172.21.15.12:0/666612729 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b0359e640 1 --2- 172.21.15.12:0/666612729 >> v2:172.21.15.12:3300/0 conn(0x7f3b0409db50 0x7f3b04141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b02d9d640 1 --2- 172.21.15.12:0/666612729 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3b040b69a0 0x7f3b04144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b01d9b640 1 -- 172.21.15.12:0/666612729 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 580008483 0 0) 0x7f3b04144870 con 0x7f3b04141670 2024-04-27T07:05:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b01d9b640 1 -- 172.21.15.12:0/666612729 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3af0003450 con 0x7f3b04141670 2024-04-27T07:05:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b0359e640 1 -- 172.21.15.12:0/666612729 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3b040b69a0 msgr2=0x7f3b04144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b0359e640 1 --2- 172.21.15.12:0/666612729 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3b040b69a0 0x7f3b04144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b0359e640 1 -- 172.21.15.12:0/666612729 >> v1:172.21.15.12:6789/0 conn(0x7f3b04141670 legacy=0x7f3b0409d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b0359e640 1 -- 172.21.15.12:0/666612729 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b040b6d80 con 0x7f3b0409db50 2024-04-27T07:05:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.192+0000 7f3b02d9d640 1 --2- 172.21.15.12:0/666612729 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3b040b69a0 0x7f3b04144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T07:05:52.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b0359e640 1 --2- 172.21.15.12:0/666612729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 0x7f3b04141ad0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0097c0 tx=0x7f3afc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e71dfce12ae432 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:52.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b01d9b640 1 -- 172.21.15.12:0/666612729 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0b0070 con 0x7f3b0409db50 2024-04-27T07:05:52.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b01d9b640 1 -- 172.21.15.12:0/666612729 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3afc0ab600 con 0x7f3b0409db50 2024-04-27T07:05:52.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b01d9b640 1 -- 172.21.15.12:0/666612729 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0ab9d0 con 0x7f3b0409db50 2024-04-27T07:05:52.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- 172.21.15.12:0/666612729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 msgr2=0x7f3b04141ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 --2- 172.21.15.12:0/666612729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 0x7f3b04141ad0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0097c0 tx=0x7f3afc01c3f0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- 172.21.15.12:0/666612729 shutdown_connections 2024-04-27T07:05:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 --2- 172.21.15.12:0/666612729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 0x7f3b04141ad0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 --2- 172.21.15.12:0/666612729 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3b040b69a0 0x7f3b04144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- 172.21.15.12:0/666612729 >> 172.21.15.12:0/666612729 conn(0x7f3b040b93b0 msgr2=0x7f3b040b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- 172.21.15.12:0/666612729 shutdown_connections 2024-04-27T07:05:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- 172.21.15.12:0/666612729 wait complete. 2024-04-27T07:05:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 Processor -- start 2024-04-27T07:05:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- start start 2024-04-27T07:05:52.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 0x7f3b041431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b040b69a0 0x7f3b040b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3b04141670 0x7f3b041dc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b040b9ea0 con 0x7f3b0409db50 2024-04-27T07:05:52.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041450c0 con 0x7f3b040b69a0 2024-04-27T07:05:52.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7f3b040ba020 con 0x7f3b04141670 2024-04-27T07:05:52.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b02d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 0x7f3b041431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:52.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b02d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 0x7f3b041431b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37070/0 (socket says 172.21.15.12:37070) 2024-04-27T07:05:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b02d9d640 1 -- 172.21.15.12:0/209011919 learned_addr learned my addr 172.21.15.12:0/209011919 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b02d9d640 1 -- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3b04141670 msgr2=0x7f3b041dc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b02d9d640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3b04141670 0x7f3b041dc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b02d9d640 1 -- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b040b69a0 msgr2=0x7f3b040b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b02d9d640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b040b69a0 0x7f3b040b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b02d9d640 1 -- 172.21.15.12:0/209011919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b040b25c0 con 0x7f3b0409db50 2024-04-27T07:05:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b02d9d640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 0x7f3b041431b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009fc0 tx=0x7f3aec00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3afb7fe640 1 -- 172.21.15.12:0/209011919 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec01a070 con 0x7f3b0409db50 2024-04-27T07:05:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3afb7fe640 1 -- 172.21.15.12:0/209011919 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aec003ba0 con 0x7f3b0409db50 2024-04-27T07:05:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3afb7fe640 1 -- 172.21.15.12:0/209011919 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004030 con 0x7f3b0409db50 2024-04-27T07:05:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- 172.21.15.12:0/209011919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b040b28a0 con 0x7f3b0409db50 2024-04-27T07:05:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.196+0000 7f3b091ef640 1 -- 172.21.15.12:0/209011919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b040afaf0 con 0x7f3b0409db50 2024-04-27T07:05:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.200+0000 7f3b091ef640 1 -- 172.21.15.12:0/209011919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b04068c20 con 0x7f3b0409db50 2024-04-27T07:05:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.200+0000 7f3afb7fe640 1 -- 172.21.15.12:0/209011919 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3aec01d070 con 0x7f3b0409db50 2024-04-27T07:05:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.200+0000 7f3afb7fe640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ae4068380 0x7f3ae406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.200+0000 7f3afb7fe640 1 -- 172.21.15.12:0/209011919 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f3aec0869a0 con 0x7f3b0409db50 2024-04-27T07:05:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.200+0000 7f3b0259c640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ae4068380 0x7f3ae406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.200+0000 7f3b0259c640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ae4068380 0x7f3ae406a840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3af4007fd0 tx=0x7f3af4007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:52.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.204+0000 7f3afb7fe640 1 -- 172.21.15.12:0/209011919 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3aec010070 con 0x7f3b0409db50 2024-04-27T07:05:52.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.364+0000 7f3b091ef640 1 -- 172.21.15.12:0/209011919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f3b04069330 con 0x7f3b0409db50 2024-04-27T07:05:52.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3afb7fe640 1 -- 172.21.15.12:0/209011919 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f3aec056b70 con 0x7f3b0409db50 2024-04-27T07:05:52.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3afb7fe640 1 -- 172.21.15.12:0/209011919 <== mon.0 v2:172.21.15.12:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3aec08c080 con 0x7f3b0409db50 2024-04-27T07:05:52.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 -- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ae4068380 msgr2=0x7f3ae406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ae4068380 0x7f3ae406a840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3af4007fd0 tx=0x7f3af4007900 comp rx=0 tx=0).stop 2024-04-27T07:05:52.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 -- 172.21.15.12:0/209011919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 msgr2=0x7f3b041431b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 0x7f3b041431b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009fc0 tx=0x7f3aec00d560 comp rx=0 tx=0).stop 2024-04-27T07:05:52.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 -- 172.21.15.12:0/209011919 shutdown_connections 2024-04-27T07:05:52.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7f3ae4068380 0x7f3ae406a840 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f3b04141670 0x7f3b041dc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b040b69a0 0x7f3b040b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 --2- 172.21.15.12:0/209011919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b0409db50 0x7f3b041431b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 -- 172.21.15.12:0/209011919 >> 172.21.15.12:0/209011919 conn(0x7f3b040b93b0 msgr2=0x7f3b04073b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:52.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 -- 172.21.15.12:0/209011919 shutdown_connections 2024-04-27T07:05:52.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.372+0000 7f3b091ef640 1 -- 172.21.15.12:0/209011919 wait complete. 2024-04-27T07:05:52.397 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-27T07:05:52.398 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T07:05:52.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc832b43640 1 Processor -- start 2024-04-27T07:05:52.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc832b43640 1 -- start start 2024-04-27T07:05:52.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc832b43640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 0x7fc82c145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc832b43640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc82c1460b0 0x7fc82c150410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc832b43640 1 -- --> v1:172.21.15.53:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc82c06dac0 con 0x7fc82c0b9a10 2024-04-27T07:05:52.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc832b43640 1 -- --> v2:172.21.15.53:3300/0 -- mon_getmap magic: 0 -- 0x7fc82c06e890 con 0x7fc82c1460b0 2024-04-27T07:05:52.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc832b43640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc82c0b6cd0 con 0x7fc82c0b3680 2024-04-27T07:05:52.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc8310e0640 1 --2- >> v2:172.21.15.53:3300/0 conn(0x7fc82c1460b0 0x7fc82c150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:52.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc82bfff640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 0x7fc82c145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:52.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc8308df640 1 --1- >> v1:172.21.15.53:6789/0 conn(0x7fc82c0b9a10 0x7fc82c0b65c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.53:6789/0 says I am v1:172.21.15.12:47188/0 (socket says 172.21.15.12:47188) 2024-04-27T07:05:52.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc8308df640 1 -- 172.21.15.12:0/2044633014 learned_addr learned my addr 172.21.15.12:0/2044633014 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-04-27T07:05:52.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc82bfff640 1 -- 172.21.15.12:0/2044633014 >> v2:172.21.15.53:3300/0 conn(0x7fc82c1460b0 msgr2=0x7fc82c150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T07:05:52.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc82bfff640 1 --2- 172.21.15.12:0/2044633014 >> v2:172.21.15.53:3300/0 conn(0x7fc82c1460b0 0x7fc82c150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc82bfff640 1 -- 172.21.15.12:0/2044633014 >> v1:172.21.15.53:6789/0 conn(0x7fc82c0b9a10 legacy=0x7fc82c0b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.508+0000 7fc82bfff640 1 -- 172.21.15.12:0/2044633014 --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c0b9df0 con 0x7fc82c0b3680 2024-04-27T07:05:52.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc82bfff640 1 --2- 172.21.15.12:0/2044633014 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 0x7fc82c145b70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc8200097c0 tx=0x7fc82001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=996ea553f8982f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc82b7fe640 1 -- 172.21.15.12:0/2044633014 <== mon.2 v2:172.21.15.53:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200b0070 con 0x7fc82c0b3680 2024-04-27T07:05:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc82b7fe640 1 -- 172.21.15.12:0/2044633014 <== mon.2 v2:172.21.15.53:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc8200ab600 con 0x7fc82c0b3680 2024-04-27T07:05:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc82b7fe640 1 -- 172.21.15.12:0/2044633014 <== mon.2 v2:172.21.15.53:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200ab9f0 con 0x7fc82c0b3680 2024-04-27T07:05:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc832b43640 1 -- 172.21.15.12:0/2044633014 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 msgr2=0x7fc82c145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc832b43640 1 --2- 172.21.15.12:0/2044633014 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 0x7fc82c145b70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc8200097c0 tx=0x7fc82001c410 comp rx=0 tx=0).stop 2024-04-27T07:05:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc832b43640 1 -- 172.21.15.12:0/2044633014 shutdown_connections 2024-04-27T07:05:52.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc832b43640 1 --2- 172.21.15.12:0/2044633014 >> v2:172.21.15.53:3300/0 conn(0x7fc82c1460b0 0x7fc82c150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc832b43640 1 --2- 172.21.15.12:0/2044633014 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 0x7fc82c145b70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc832b43640 1 -- 172.21.15.12:0/2044633014 >> 172.21.15.12:0/2044633014 conn(0x7fc82c067d10 msgr2=0x7fc82c068120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:52.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc832b43640 1 -- 172.21.15.12:0/2044633014 shutdown_connections 2024-04-27T07:05:52.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.512+0000 7fc832b43640 1 -- 172.21.15.12:0/2044633014 wait complete. 2024-04-27T07:05:52.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 Processor -- start 2024-04-27T07:05:52.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 -- start start 2024-04-27T07:05:52.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 --2- >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 0x7fc82c13f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc82c0b9a10 0x7fc82c0b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc82c1460b0 0x7fc82c13acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c074020 con 0x7fc82c0b9a10 2024-04-27T07:05:52.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c073ea0 con 0x7fc82c1460b0 2024-04-27T07:05:52.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 -- --> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] -- mon_getmap magic: 0 -- 0x7fc82c153170 con 0x7fc82c0b3680 2024-04-27T07:05:52.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc82bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc82c0b9a10 0x7fc82c0b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:52.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc82bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc82c0b9a10 0x7fc82c0b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37086/0 (socket says 172.21.15.12:37086) 2024-04-27T07:05:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc82bfff640 1 -- 172.21.15.12:0/2200815619 learned_addr learned my addr 172.21.15.12:0/2200815619 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-27T07:05:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc8308df640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 0x7fc82c13f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc8310e0640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc82c1460b0 0x7fc82c13acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc82bfff640 1 -- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 msgr2=0x7fc82c13f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc82bfff640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 0x7fc82c13f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc82bfff640 1 -- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc82c1460b0 msgr2=0x7fc82c13acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc82bfff640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc82c1460b0 0x7fc82c13acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc82bfff640 1 -- 172.21.15.12:0/2200815619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c0b09c0 con 0x7fc82c0b9a10 2024-04-27T07:05:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc82bfff640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc82c0b9a10 0x7fc82c0b10c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc82001a040 tx=0x7fc8200191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc8297fa640 1 -- 172.21.15.12:0/2200815619 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200b8020 con 0x7fc82c0b9a10 2024-04-27T07:05:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc8297fa640 1 -- 172.21.15.12:0/2200815619 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc8200b4410 con 0x7fc82c0b9a10 2024-04-27T07:05:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc8297fa640 1 -- 172.21.15.12:0/2200815619 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200b0040 con 0x7fc82c0b9a10 2024-04-27T07:05:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 -- 172.21.15.12:0/2200815619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82c0b28d0 con 0x7fc82c0b9a10 2024-04-27T07:05:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 -- 172.21.15.12:0/2200815619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82c13fe00 con 0x7fc82c0b9a10 2024-04-27T07:05:52.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc832b43640 1 -- 172.21.15.12:0/2200815619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82c068c20 con 0x7fc82c0b9a10 2024-04-27T07:05:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.516+0000 7fc8297fa640 1 -- 172.21.15.12:0/2200815619 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc8200bb020 con 0x7fc82c0b9a10 2024-04-27T07:05:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.520+0000 7fc8297fa640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc818068380 0x7fc81806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T07:05:52.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.520+0000 7fc8297fa640 1 -- 172.21.15.12:0/2200815619 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fc8201202a0 con 0x7fc82c0b9a10 2024-04-27T07:05:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.520+0000 7fc8308df640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc818068380 0x7fc81806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T07:05:52.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.520+0000 7fc8308df640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc818068380 0x7fc81806a840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc81c000f70 tx=0x7fc81c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T07:05:52.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.524+0000 7fc8297fa640 1 -- 172.21.15.12:0/2200815619 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc8200c2030 con 0x7fc82c0b9a10 2024-04-27T07:05:52.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 -- 172.21.15.12:0/2200815619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc82c145950 con 0x7fc82c0b9a10 2024-04-27T07:05:52.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc8297fa640 1 -- 172.21.15.12:0/2200815619 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+2216 (secure 0 0 0) 0x7fc820016090 con 0x7fc82c0b9a10 2024-04-27T07:05:52.730 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-27T07:05:52.730 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4613,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/3153112684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":3153112684},{"type":"v1","addr":"172.21.15.53:6835","nonce":3153112684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6833/474218025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":474218025},{"type":"v1","addr":"172.21.15.12:6833","nonce":474218025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-04-27T07:05:52.730 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 18 2024-04-27T07:05:52.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 -- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc818068380 msgr2=0x7fc81806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc818068380 0x7fc81806a840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc81c000f70 tx=0x7fc81c009040 comp rx=0 tx=0).stop 2024-04-27T07:05:52.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 -- 172.21.15.12:0/2200815619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc82c0b9a10 msgr2=0x7fc82c0b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T07:05:52.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc82c0b9a10 0x7fc82c0b10c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc82001a040 tx=0x7fc8200191b0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 -- 172.21.15.12:0/2200815619 shutdown_connections 2024-04-27T07:05:52.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:6832/452515146,v1:172.21.15.53:6833/452515146] conn(0x7fc818068380 0x7fc81806a840 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc82c1460b0 0x7fc82c13acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc82c0b9a10 0x7fc82c0b10c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 --2- 172.21.15.12:0/2200815619 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7fc82c0b3680 0x7fc82c13f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T07:05:52.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 -- 172.21.15.12:0/2200815619 >> 172.21.15.12:0/2200815619 conn(0x7fc82c067d10 msgr2=0x7fc82c0b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T07:05:52.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 -- 172.21.15.12:0/2200815619 shutdown_connections 2024-04-27T07:05:52.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-27T07:05:52.724+0000 7fc832b43640 1 -- 172.21.15.12:0/2200815619 wait complete. 2024-04-27T07:05:52.748 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-27T07:05:52.749 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-27T07:05:52.749 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-27T07:05:52.749 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:52.760 INFO:tasks.ceph.mds.a:Stopped 2024-04-27T07:05:52.760 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-27T07:05:52.760 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:52.776 INFO:tasks.ceph.mds.b:Stopped 2024-04-27T07:05:52.776 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-27T07:05:52.777 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-27T07:05:52.777 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:55.104 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:05:55.107 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:05:58.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-04-27T07:05:58.879 INFO:tasks.ceph.osd.0:Stopped 2024-04-27T07:05:58.879 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-27T07:05:58.880 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:58.952 INFO:tasks.ceph.osd.1:Stopped 2024-04-27T07:05:58.952 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-27T07:05:58.952 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.045 INFO:tasks.ceph.osd.2:Stopped 2024-04-27T07:05:59.045 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-27T07:05:59.045 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.127 INFO:tasks.ceph.osd.3:Stopped 2024-04-27T07:05:59.127 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-27T07:05:59.127 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.206 INFO:tasks.ceph.osd.4:Stopped 2024-04-27T07:05:59.206 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-27T07:05:59.206 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.280 INFO:tasks.ceph.osd.5:Stopped 2024-04-27T07:05:59.281 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-27T07:05:59.281 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.363 INFO:tasks.ceph.osd.6:Stopped 2024-04-27T07:05:59.363 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-27T07:05:59.363 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.443 INFO:tasks.ceph.osd.7:Stopped 2024-04-27T07:05:59.444 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-27T07:05:59.444 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-27T07:05:59.444 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.473 INFO:tasks.ceph.mgr.y:Stopped 2024-04-27T07:05:59.473 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-27T07:05:59.473 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.502 INFO:tasks.ceph.mgr.x:Stopped 2024-04-27T07:05:59.503 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-27T07:05:59.503 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-27T07:05:59.503 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.535 INFO:tasks.ceph.mon.a:Stopped 2024-04-27T07:05:59.535 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-27T07:05:59.535 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.568 INFO:tasks.ceph.mon.b:Stopped 2024-04-27T07:05:59.569 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-27T07:05:59.569 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T07:05:59.624 INFO:tasks.ceph.mon.c:Stopped 2024-04-27T07:05:59.625 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-27T07:05:59.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-27T07:05:59.644 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:05:59.644 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-27T07:05:59.848 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:05:59.849 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-27T07:05:59.985 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:05:59.985 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-27T07:06:00.111 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:06:00.111 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-27T07:06:00.237 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T07:06:00.238 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-27T07:06:00.471 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T07:06:00.471 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-27T07:06:00.667 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T07:06:00.667 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-27T07:06:00.816 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T07:06:00.816 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-27T07:06:00.976 INFO:tasks.ceph:Archiving mon data... 2024-04-27T07:06:00.978 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037/data/mon.a.tgz 2024-04-27T07:06:00.978 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-04-27T07:06:00.984 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.U3iVWzJX7f 2024-04-27T07:06:00.984 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.U3iVWzJX7f 2024-04-27T07:06:01.103 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.U3iVWzJX7f 2024-04-27T07:06:01.170 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.U3iVWzJX7f is 270KB 2024-04-27T07:06:01.231 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.U3iVWzJX7f 2024-04-27T07:06:01.236 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037/data/mon.b.tgz 2024-04-27T07:06:01.236 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-04-27T07:06:01.242 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.QipiuoanjW 2024-04-27T07:06:01.243 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.QipiuoanjW 2024-04-27T07:06:01.371 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.QipiuoanjW 2024-04-27T07:06:01.434 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.QipiuoanjW is 366KB 2024-04-27T07:06:01.497 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.QipiuoanjW 2024-04-27T07:06:01.502 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037/data/mon.c.tgz 2024-04-27T07:06:01.502 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-04-27T07:06:01.555 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.XUmjoPO8Yz 2024-04-27T07:06:01.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.XUmjoPO8Yz 2024-04-27T07:06:01.683 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.XUmjoPO8Yz 2024-04-27T07:06:01.746 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.XUmjoPO8Yz is 366KB 2024-04-27T07:06:01.765 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.XUmjoPO8Yz 2024-04-27T07:06:01.769 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-27T07:06:01.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-27T07:06:01.772 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-27T07:06:01.833 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-27T07:06:01.871 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-27T07:06:01.871 INFO:tasks.ceph:Archiving crash dumps... 2024-04-27T07:06:01.876 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037/remote/smithi012/crash 2024-04-27T07:06:01.877 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-27T07:06:01.891 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037/remote/smithi053/crash 2024-04-27T07:06:01.893 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-27T07:06:01.927 INFO:tasks.ceph:Shutting down logrotate 2024-04-27T07:06:01.928 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:06:01.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:06:02.015 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-27T07:06:02.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-27T07:06:02.074 INFO:tasks.ceph:Compressing logs... 2024-04-27T07:06:02.074 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T07:06:02.078 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T07:06:02.103 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37719.log 2024-04-27T07:06:02.103 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35988.log 2024-04-27T07:06:02.103 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37347.log 2024-04-27T07:06:02.105 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34090.log 2024-04-27T07:06:02.105 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37347.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35988.log.gz 2024-04-27T07:06:02.105 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37719.log.gz 2024-04-27T07:06:02.105 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35619.log 2024-04-27T07:06:02.106 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37347.log.gz 2024-04-27T07:06:02.106 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36344.log 2024-04-27T07:06:02.106 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34090.log.gz 2024-04-27T07:06:02.109 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35299.log 2024-04-27T07:06:02.109 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35619.log: /var/log/ceph/ceph-client.admin.36344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34091.log 2024-04-27T07:06:02.109 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35619.log.gz 2024-04-27T07:06:02.109 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35299.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36344.log.gz 2024-04-27T07:06:02.109 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37056.log 2024-04-27T07:06:02.109 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35299.log.gz 2024-04-27T07:06:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5/var/log/ceph/ceph-client.admin.34091.log: --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-27T07:06:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37056.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34091.log.gz 2024-04-27T07:06:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33687.log 2024-04-27T07:06:02.110 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37056.log.gz 2024-04-27T07:06:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34673.log 2024-04-27T07:06:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35039.log 2024-04-27T07:06:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35247.log 2024-04-27T07:06:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34673.log: /var/log/ceph/ceph-client.admin.35039.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33687.log.gz 2024-04-27T07:06:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35962.log 2024-04-27T07:06:02.111 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35039.log.gz 2024-04-27T07:06:02.117 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37771.log 2024-04-27T07:06:02.117 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35247.log: /var/log/ceph/ceph-client.admin.35962.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.34673.log.gzgzip 2024-04-27T07:06:02.117 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33683.log 2024-04-27T07:06:02.117 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35247.log.gz 2024-04-27T07:06:02.117 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35962.log.gz 2024-04-27T07:06:02.117 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35429.log 2024-04-27T07:06:02.117 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37771.log.gz 2024-04-27T07:06:02.118 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36568.log 2024-04-27T07:06:02.118 INFO:teuthology.orchestra.run.smithi012.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33683.log.gz 2024-04-27T07:06:02.118 INFO:teuthology.orchestra.run.smithi012.stderr: 92.5%gzip -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-27T07:06:02.118 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34987.log 2024-04-27T07:06:02.118 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35429.log: /var/log/ceph/ceph-client.admin.36568.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-27T07:06:02.118 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35429.log.gz 2024-04-27T07:06:02.118 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34987.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36568.log.gz 2024-04-27T07:06:02.118 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36728.log 2024-04-27T07:06:02.119 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34987.log.gz 2024-04-27T07:06:02.119 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36422.log 2024-04-27T07:06:02.119 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33895.log 2024-04-27T07:06:02.119 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36728.log: /var/log/ceph/ceph-client.admin.36422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37875.log 2024-04-27T07:06:02.119 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36728.log.gz 2024-04-27T07:06:02.119 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33895.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36422.log.gz 2024-04-27T07:06:02.119 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37108.log 2024-04-27T07:06:02.120 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33895.log.gz 2024-04-27T07:06:02.120 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27535.log 2024-04-27T07:06:02.120 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36912.log 2024-04-27T07:06:02.120 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37875.log.gz 2024-04-27T07:06:02.120 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37108.log.gz 2024-04-27T07:06:02.120 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.27535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37426.log 2024-04-27T07:06:02.120 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27535.log.gz 2024-04-27T07:06:02.120 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.0.36912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37667.log 2024-04-27T07:06:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35351.log 2024-04-27T07:06:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37693.log 2024-04-27T07:06:02.121 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37426.log.gz 2024-04-27T07:06:02.121 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35351.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37667.log.gz 2024-04-27T07:06:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33884.log 2024-04-27T07:06:02.122 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35351.log.gz 2024-04-27T07:06:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36187.log 2024-04-27T07:06:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33884.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.37693.log.gz 2024-04-27T07:06:02.122 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37523.log 2024-04-27T07:06:02.141 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33884.log.gz 2024-04-27T07:06:02.142 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37082.log 2024-04-27T07:06:02.142 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36187.log.gz 2024-04-27T07:06:02.142 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35488.log 2024-04-27T07:06:02.142 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35671.log 2024-04-27T07:06:02.142 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37523.log.gz 2024-04-27T07:06:02.142 INFO:teuthology.orchestra.run.smithi012.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.36912.log.gz 2024-04-27T07:06:02.142 INFO:teuthology.orchestra.run.smithi012.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37082.log.gz 2024-04-27T07:06:02.142 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33315.log 2024-04-27T07:06:02.143 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35488.log.gz 2024-04-27T07:06:02.143 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34491.log 2024-04-27T07:06:02.143 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33315.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-27T07:06:02.143 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35671.log.gz 2024-04-27T07:06:02.143 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33315.log.gz 2024-04-27T07:06:02.143 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33227.log 2024-04-27T07:06:02.143 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34491.log.gz 2024-04-27T07:06:02.143 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33681.log 2024-04-27T07:06:02.143 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33199.log 2024-04-27T07:06:02.144 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33681.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33227.log.gz 2024-04-27T07:06:02.144 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33119.log 2024-04-27T07:06:02.144 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33681.log.gz 2024-04-27T07:06:02.144 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33607.log 2024-04-27T07:06:02.144 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33199.log: /var/log/ceph/ceph-client.admin.33119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36856.log 2024-04-27T07:06:02.144 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33119.log.gz 2024-04-27T07:06:02.144 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33199.log.gz/var/log/ceph/ceph-client.admin.33607.log: 2024-04-27T07:06:02.144 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37321.log 2024-04-27T07:06:02.144 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33607.log.gz 2024-04-27T07:06:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34387.log 2024-04-27T07:06:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37321.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-27T07:06:02.145 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36856.log.gz 2024-04-27T07:06:02.145 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37321.log.gz 2024-04-27T07:06:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37953.log 2024-04-27T07:06:02.145 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34387.log.gz 2024-04-27T07:06:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34961.log 2024-04-27T07:06:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.37953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34102.log 2024-04-27T07:06:02.145 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34961.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37953.log.gz 2024-04-27T07:06:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35592.log 2024-04-27T07:06:02.146 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34961.log.gz 2024-04-27T07:06:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36266.log 2024-04-27T07:06:02.146 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34102.log.gz 2024-04-27T07:06:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35403.log 2024-04-27T07:06:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35592.log: /var/log/ceph/ceph-client.admin.36266.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35592.log.gz 2024-04-27T07:06:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35910.log 2024-04-27T07:06:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35403.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36266.log.gz 2024-04-27T07:06:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34465.log 2024-04-27T07:06:02.147 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35403.log.gz 2024-04-27T07:06:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37927.log 2024-04-27T07:06:02.147 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35910.log.gz 2024-04-27T07:06:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.34465.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37191.log 2024-04-27T07:06:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33263.log 2024-04-27T07:06:02.147 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34465.log.gz 2024-04-27T07:06:02.147 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37927.log.gz 2024-04-27T07:06:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37797.log 2024-04-27T07:06:02.148 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37191.log.gz 2024-04-27T07:06:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33449.log 2024-04-27T07:06:02.148 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33263.log.gz/var/log/ceph/ceph-client.admin.37797.log: 2024-04-27T07:06:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37269.log 2024-04-27T07:06:02.148 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37797.log.gz 2024-04-27T07:06:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34728.log 2024-04-27T07:06:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33449.log: /var/log/ceph/ceph-client.admin.37269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35195.log 2024-04-27T07:06:02.148 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33449.log.gz 2024-04-27T07:06:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34728.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37269.log.gz 2024-04-27T07:06:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33880.log 2024-04-27T07:06:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35195.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34728.log.gz 2024-04-27T07:06:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36085.log 2024-04-27T07:06:02.149 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35195.log.gz 2024-04-27T07:06:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37217.log 2024-04-27T07:06:02.149 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33880.log.gz 2024-04-27T07:06:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35723.log 2024-04-27T07:06:02.149 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36085.log.gz 2024-04-27T07:06:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34595.log 2024-04-27T07:06:02.150 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37217.log.gz 2024-04-27T07:06:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36504.log 2024-04-27T07:06:02.150 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35723.log.gz 2024-04-27T07:06:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37589.log 2024-04-27T07:06:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36504.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34595.log.gz 2024-04-27T07:06:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33423.log 2024-04-27T07:06:02.150 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36504.log.gz 2024-04-27T07:06:02.150 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37471.log 2024-04-27T07:06:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33423.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37589.log.gz 2024-04-27T07:06:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37400.log 2024-04-27T07:06:02.151 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33423.log.gz 2024-04-27T07:06:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33289.log 2024-04-27T07:06:02.151 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37471.log.gz 2024-04-27T07:06:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34361.log 2024-04-27T07:06:02.151 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37400.log.gz 2024-04-27T07:06:02.151 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36040.log 2024-04-27T07:06:02.152 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33289.log.gz 2024-04-27T07:06:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33682.log 2024-04-27T07:06:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34361.log: /var/log/ceph/ceph-client.admin.36040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33501.log 2024-04-27T07:06:02.152 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34361.log.gz 2024-04-27T07:06:02.152 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36040.log.gz 2024-04-27T07:06:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34569.log 2024-04-27T07:06:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35169.log 2024-04-27T07:06:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33501.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33682.log.gz 2024-04-27T07:06:02.152 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34569.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33501.log.gz 2024-04-27T07:06:02.153 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34309.log 2024-04-27T07:06:02.153 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34569.log.gz 2024-04-27T07:06:02.153 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.35169.log: -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-27T07:06:02.153 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36396.log 2024-04-27T07:06:02.153 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35169.log.gz 2024-04-27T07:06:02.153 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.3.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34309.log.gz 2024-04-27T07:06:02.153 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35377.log 2024-04-27T07:06:02.153 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37979.log 2024-04-27T07:06:02.153 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36396.log.gz 2024-04-27T07:06:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35645.log 2024-04-27T07:06:02.154 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9%/var/log/ceph/ceph-client.admin.37979.log: -- replaced with /var/log/ceph/ceph-client.admin.35377.log.gz 2024-04-27T07:06:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34283.log 2024-04-27T07:06:02.154 INFO:teuthology.orchestra.run.smithi012.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37979.log.gz 2024-04-27T07:06:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36476.log 2024-04-27T07:06:02.154 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1%/var/log/ceph/ceph-client.admin.34283.log: -- replaced with /var/log/ceph/ceph-client.admin.35645.log.gz 2024-04-27T07:06:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36594.log 2024-04-27T07:06:02.154 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36476.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34283.log.gz 2024-04-27T07:06:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35143.log 2024-04-27T07:06:02.155 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36476.log.gz 2024-04-27T07:06:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34543.log 2024-04-27T07:06:02.155 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1%/var/log/ceph/ceph-client.admin.35143.log: -- replaced with /var/log/ceph/ceph-client.admin.36594.log.gz 2024-04-27T07:06:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36318.log 2024-04-27T07:06:02.155 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35143.log.gz 2024-04-27T07:06:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35540.log 2024-04-27T07:06:02.155 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34543.log.gz 2024-04-27T07:06:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36624.log 2024-04-27T07:06:02.166 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-27T07:06:02.166 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27635.log 2024-04-27T07:06:02.166 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-27T07:06:02.166 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.27635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27539.log 2024-04-27T07:06:02.166 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-27T07:06:02.166 INFO:teuthology.orchestra.run.smithi053.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27635.log.gz 2024-04-27T07:06:02.167 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.27539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27611.log 2024-04-27T07:06:02.167 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.5.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27539.log.gz 2024-04-27T07:06:02.167 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27563.log 2024-04-27T07:06:02.167 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27587.log 2024-04-27T07:06:02.167 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.27611.log: /var/log/ceph/ceph-client.admin.27563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27707.log 2024-04-27T07:06:02.167 INFO:teuthology.orchestra.run.smithi053.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27611.log.gz 2024-04-27T07:06:02.167 INFO:teuthology.orchestra.run.smithi053.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27563.log.gz 2024-04-27T07:06:02.167 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27683.log 2024-04-27T07:06:02.167 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.27587.log: /var/log/ceph/ceph-client.admin.27707.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-27T07:06:02.168 INFO:teuthology.orchestra.run.smithi053.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27587.log.gz 2024-04-27T07:06:02.168 INFO:teuthology.orchestra.run.smithi053.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27707.log.gz 2024-04-27T07:06:02.168 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.27683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27659.log 2024-04-27T07:06:02.168 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mgr.x.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27683.log.gz 2024-04-27T07:06:02.168 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-27T07:06:02.168 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.27659.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-27T07:06:02.168 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log/var/log/ceph/ceph.log: 2024-04-27T07:06:02.168 INFO:teuthology.orchestra.run.smithi053.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27659.log.gz 2024-04-27T07:06:02.168 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-27T07:06:02.169 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-27T07:06:02.169 INFO:teuthology.orchestra.run.smithi053.stderr: 91.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-27T07:06:02.169 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-27T07:06:02.169 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mon.b.log: 91.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-27T07:06:02.169 INFO:teuthology.orchestra.run.smithi053.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-27T07:06:02.185 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.4.log: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-27T07:06:02.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35540.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36318.log.gz 2024-04-27T07:06:02.188 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37243.log 2024-04-27T07:06:02.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36624.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35540.log.gz 2024-04-27T07:06:02.189 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-27T07:06:02.189 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36624.log.gz 2024-04-27T07:06:02.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36780.log 2024-04-27T07:06:02.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.1.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37243.log.gz 2024-04-27T07:06:02.189 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37497.log 2024-04-27T07:06:02.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35273.log 2024-04-27T07:06:02.189 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7%/var/log/ceph/ceph-client.admin.37497.log: -- replaced with /var/log/ceph/ceph-client.admin.36780.log.gz 2024-04-27T07:06:02.189 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35936.log 2024-04-27T07:06:02.190 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37497.log.gz 2024-04-27T07:06:02.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35273.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35273.log.gz 2024-04-27T07:06:02.190 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37901.log 2024-04-27T07:06:02.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36448.log 2024-04-27T07:06:02.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37901.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35936.log.gz 2024-04-27T07:06:02.190 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36702.log 2024-04-27T07:06:02.190 INFO:teuthology.orchestra.run.smithi012.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37901.log.gz 2024-04-27T07:06:02.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36448.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36448.log.gz 2024-04-27T07:06:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36806.log 2024-04-27T07:06:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33882.log 2024-04-27T07:06:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36806.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38005.log 2024-04-27T07:06:02.191 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36702.log.gz 2024-04-27T07:06:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33882.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.36806.log.gz 2024-04-27T07:06:02.191 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33882.log.gz 2024-04-27T07:06:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38185.log 2024-04-27T07:06:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35566.log 2024-04-27T07:06:02.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38185.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38005.log.gz 2024-04-27T07:06:02.192 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36650.log 2024-04-27T07:06:02.192 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38185.log.gz 2024-04-27T07:06:02.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35566.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35566.log.gz 2024-04-27T07:06:02.192 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33171.log 2024-04-27T07:06:02.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33011.log 2024-04-27T07:06:02.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33171.log: gzip 89.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.36650.log.gz -- 2024-04-27T07:06:02.192 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.33555.log 2024-04-27T07:06:02.192 INFO:teuthology.orchestra.run.smithi012.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33171.log.gz 2024-04-27T07:06:02.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33011.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33011.log.gz 2024-04-27T07:06:02.193 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35325.log 2024-04-27T07:06:02.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37295.log 2024-04-27T07:06:02.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35325.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33555.log.gz 2024-04-27T07:06:02.193 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35514.log 2024-04-27T07:06:02.193 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35325.log.gz 2024-04-27T07:06:02.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37295.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37295.log.gz 2024-04-27T07:06:02.193 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37823.log 2024-04-27T07:06:02.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33397.log 2024-04-27T07:06:02.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37823.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35514.log.gz 2024-04-27T07:06:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27645.log 2024-04-27T07:06:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-27T07:06:02.194 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37823.log.gz/var/log/ceph/ceph-client.admin.33397.log: 2024-04-27T07:06:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.27645.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33397.log.gz 2024-04-27T07:06:02.194 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27645.log.gz 2024-04-27T07:06:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33475.log 2024-04-27T07:06:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36973.log 2024-04-27T07:06:02.194 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37004.log 2024-04-27T07:06:02.195 INFO:teuthology.orchestra.run.smithi012.stderr: 84.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-27T07:06:02.195 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33475.log.gz/var/log/ceph/ceph-client.admin.36973.log: 2024-04-27T07:06:02.195 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36973.log.gz 2024-04-27T07:06:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33581.log 2024-04-27T07:06:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38159.log 2024-04-27T07:06:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33581.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.37004.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.35117.log 2024-04-27T07:06:02.195 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-27T07:06:02.195 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33581.log.gz 2024-04-27T07:06:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38159.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38159.log.gz 2024-04-27T07:06:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34439.log 2024-04-27T07:06:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33685.log 2024-04-27T07:06:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34439.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35117.log.gz 2024-04-27T07:06:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35455.log 2024-04-27T07:06:02.196 INFO:teuthology.orchestra.run.smithi012.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34439.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36014.log 2024-04-27T07:06:02.196 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-27T07:06:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33685.log: /var/log/ceph/ceph-client.admin.35455.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35455.log.gz 2024-04-27T07:06:02.196 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33685.log.gz 2024-04-27T07:06:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33527.log 2024-04-27T07:06:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37139.log 2024-04-27T07:06:02.197 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1%/var/log/ceph/ceph-client.admin.33527.log: -- replaced with /var/log/ceph/ceph-client.admin.36014.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33345.log 2024-04-27T07:06:02.197 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-27T07:06:02.197 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33527.log.gz 2024-04-27T07:06:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37139.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37139.log.gz 2024-04-27T07:06:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37615.log 2024-04-27T07:06:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38083.log 2024-04-27T07:06:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37615.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33345.log.gz 2024-04-27T07:06:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32933.log 2024-04-27T07:06:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36239.log 2024-04-27T07:06:02.198 INFO:teuthology.orchestra.run.smithi012.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37615.log.gz 2024-04-27T07:06:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38083.log: /var/log/ceph/ceph-client.admin.32933.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38083.log.gz 2024-04-27T07:06:02.198 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32933.log.gz 2024-04-27T07:06:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34335.log 2024-04-27T07:06:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36111.log 2024-04-27T07:06:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34335.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36239.log.gz 2024-04-27T07:06:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35780.log 2024-04-27T07:06:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35091.log 2024-04-27T07:06:02.199 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2%/var/log/ceph/ceph-client.admin.36111.log: -- replaced with /var/log/ceph/ceph-client.admin.34335.log.gz 2024-04-27T07:06:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35780.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36111.log.gz 2024-04-27T07:06:02.199 INFO:teuthology.orchestra.run.smithi012.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35780.log.gz 2024-04-27T07:06:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-27T07:06:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35091.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22661.log 2024-04-27T07:06:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.audit.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35091.log.gz 2024-04-27T07:06:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38109.log 2024-04-27T07:06:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.tmp-client.admin.22661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36754.log 2024-04-27T07:06:02.199 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22661.log.gz 2024-04-27T07:06:02.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38109.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38109.log.gz 2024-04-27T07:06:02.200 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-27T07:06:02.200 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38057.log 2024-04-27T07:06:02.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37030.log 2024-04-27T07:06:02.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38057.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36754.log.gz 2024-04-27T07:06:02.200 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33093.log 2024-04-27T07:06:02.200 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-27T07:06:02.200 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38057.log.gz 2024-04-27T07:06:02.201 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37030.log: /var/log/ceph/ceph-client.admin.33093.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37030.log.gz 2024-04-27T07:06:02.201 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33093.log.gz 2024-04-27T07:06:02.201 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37374.log 2024-04-27T07:06:02.201 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34103.log 2024-04-27T07:06:02.201 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34082.log 2024-04-27T07:06:02.201 INFO:teuthology.orchestra.run.smithi012.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37374.log.gz 2024-04-27T07:06:02.201 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34103.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34103.log.gz 2024-04-27T07:06:02.201 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34413.log 2024-04-27T07:06:02.201 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34856.log 2024-04-27T07:06:02.201 INFO:teuthology.orchestra.run.smithi012.stderr: 87.9%/var/log/ceph/ceph-client.admin.34413.log: -- replaced with /var/log/ceph/ceph-client.admin.34082.log.gz 2024-04-27T07:06:02.202 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34413.log.gz 2024-04-27T07:06:02.202 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34909.log 2024-04-27T07:06:02.202 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34621.log 2024-04-27T07:06:02.202 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.0.34856.log: /var/log/ceph/ceph-client.admin.34909.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34909.log.gz 2024-04-27T07:06:02.202 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.0.34856.log.gz 2024-04-27T07:06:02.202 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33887.log 2024-04-27T07:06:02.202 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35858.log 2024-04-27T07:06:02.202 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34621.log: /var/log/ceph/ceph-client.admin.33887.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34621.log.gz 2024-04-27T07:06:02.203 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33887.log.gz 2024-04-27T07:06:02.203 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36142.log 2024-04-27T07:06:02.203 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33058.log 2024-04-27T07:06:02.203 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35858.log: /var/log/ceph/ceph-client.admin.36142.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35858.log.gz 2024-04-27T07:06:02.203 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36142.log.gz 2024-04-27T07:06:02.203 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35221.log 2024-04-27T07:06:02.203 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36676.log 2024-04-27T07:06:02.203 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33058.log: /var/log/ceph/ceph-client.admin.35221.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33058.log.gz 2024-04-27T07:06:02.203 INFO:teuthology.orchestra.run.smithi012.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35221.log.gzgzip 2024-04-27T07:06:02.204 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33145.log 2024-04-27T07:06:02.204 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36676.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36676.log.gz 2024-04-27T07:06:02.204 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35013.log 2024-04-27T07:06:02.204 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35754.log 2024-04-27T07:06:02.204 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33145.log: /var/log/ceph/ceph-client.admin.35013.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33145.log.gz 2024-04-27T07:06:02.204 INFO:teuthology.orchestra.run.smithi012.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35013.log.gz 2024-04-27T07:06:02.204 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27426.log 2024-04-27T07:06:02.216 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37745.log 2024-04-27T07:06:02.216 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35754.log: /var/log/ceph/ceph-client.admin.27426.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35754.log.gz 2024-04-27T07:06:02.216 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33680.log 2024-04-27T07:06:02.216 INFO:teuthology.orchestra.run.smithi012.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.27426.log.gz 2024-04-27T07:06:02.216 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37745.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37745.log.gz 2024-04-27T07:06:02.216 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34647.log 2024-04-27T07:06:02.217 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34517.log 2024-04-27T07:06:02.217 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33680.log: /var/log/ceph/ceph-client.admin.34647.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33680.log.gz 2024-04-27T07:06:02.217 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34647.log.gz 2024-04-27T07:06:02.217 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35697.log 2024-04-27T07:06:02.217 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34075.log 2024-04-27T07:06:02.217 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34517.log: /var/log/ceph/ceph-client.admin.35697.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34517.log.gz 2024-04-27T07:06:02.217 INFO:teuthology.orchestra.run.smithi012.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35697.log.gz 2024-04-27T07:06:02.217 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33891.log 2024-04-27T07:06:02.217 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37641.log 2024-04-27T07:06:02.218 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34075.log: /var/log/ceph/ceph-client.admin.33891.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34075.log.gz 2024-04-27T07:06:02.218 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33891.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34101.log 2024-04-27T07:06:02.218 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-27T07:06:02.218 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37641.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37641.log.gz 2024-04-27T07:06:02.218 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33883.log 2024-04-27T07:06:02.218 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38031.log 2024-04-27T07:06:02.218 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34101.log: /var/log/ceph/ceph-client.admin.33883.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34101.log.gz 2024-04-27T07:06:02.218 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.33883.log.gz 2024-04-27T07:06:02.219 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37165.log 2024-04-27T07:06:02.219 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38031.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38031.log.gz 2024-04-27T07:06:02.219 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35884.log 2024-04-27T07:06:02.219 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33679.log 2024-04-27T07:06:02.219 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37165.log: /var/log/ceph/ceph-client.admin.35884.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37165.log.gz 2024-04-27T07:06:02.219 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35884.log.gz 2024-04-27T07:06:02.219 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36213.log 2024-04-27T07:06:02.219 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33679.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33679.log.gz 2024-04-27T07:06:02.219 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33633.log 2024-04-27T07:06:02.220 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36370.log 2024-04-27T07:06:02.220 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36213.log: /var/log/ceph/ceph-client.admin.33633.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36213.log.gz 2024-04-27T07:06:02.220 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33633.log.gz 2024-04-27T07:06:02.220 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37849.log 2024-04-27T07:06:02.220 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36542.log 2024-04-27T07:06:02.220 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36370.log: /var/log/ceph/ceph-client.admin.37849.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36370.log.gz 2024-04-27T07:06:02.220 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37849.log.gz 2024-04-27T07:06:02.220 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33881.log 2024-04-27T07:06:02.220 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35832.log 2024-04-27T07:06:02.221 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36542.log: /var/log/ceph/ceph-client.admin.33881.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36542.log.gz 2024-04-27T07:06:02.221 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36292.log 2024-04-27T07:06:02.221 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33881.log.gz 2024-04-27T07:06:02.221 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35832.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35832.log.gz 2024-04-27T07:06:02.221 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35065.log 2024-04-27T07:06:02.221 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33686.log 2024-04-27T07:06:02.221 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36292.log: /var/log/ceph/ceph-client.admin.35065.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36292.log.gz 2024-04-27T07:06:02.221 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35065.log.gz 2024-04-27T07:06:02.222 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34935.log 2024-04-27T07:06:02.222 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32963.log 2024-04-27T07:06:02.222 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33686.log: /var/log/ceph/ceph-client.admin.34935.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33686.log.gz 2024-04-27T07:06:02.222 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34935.log.gz 2024-04-27T07:06:02.222 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33371.log 2024-04-27T07:06:02.222 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32963.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32963.log.gz 2024-04-27T07:06:02.222 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34089.log 2024-04-27T07:06:02.222 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37549.log 2024-04-27T07:06:02.222 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33371.log: /var/log/ceph/ceph-client.admin.34089.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33371.log.gz 2024-04-27T07:06:02.223 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34089.log.gz 2024-04-27T07:06:02.223 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35806.log 2024-04-27T07:06:02.223 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38135.log 2024-04-27T07:06:02.223 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37549.log: /var/log/ceph/ceph-client.admin.35806.log: 87.7% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37549.log.gz 2024-04-27T07:06:02.223 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35806.log.gz 2024-04-27T07:06:02.223 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38135.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38135.log.gz 2024-04-27T07:06:02.223 INFO:teuthology.orchestra.run.smithi012.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-27T07:06:02.227 INFO:teuthology.orchestra.run.smithi012.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-27T07:06:02.233 INFO:teuthology.orchestra.run.smithi012.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-27T07:06:02.243 INFO:teuthology.orchestra.run.smithi053.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-27T07:06:02.255 INFO:teuthology.orchestra.run.smithi053.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-27T07:06:02.264 INFO:teuthology.orchestra.run.smithi012.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-27T07:06:02.265 INFO:teuthology.orchestra.run.smithi053.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-27T07:06:02.272 INFO:teuthology.orchestra.run.smithi012.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-27T07:06:02.278 INFO:teuthology.orchestra.run.smithi053.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-27T07:06:02.291 INFO:teuthology.orchestra.run.smithi053.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-27T07:06:02.319 INFO:teuthology.orchestra.run.smithi053.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-27T07:06:02.319 INFO:teuthology.orchestra.run.smithi012.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-27T07:06:02.320 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-04-27T07:06:02.320 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m0.193s 2024-04-27T07:06:02.320 INFO:teuthology.orchestra.run.smithi053.stderr:user 0m0.756s 2024-04-27T07:06:02.320 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m0.054s 2024-04-27T07:06:02.321 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-27T07:06:02.322 INFO:teuthology.orchestra.run.smithi012.stderr:real 0m0.242s 2024-04-27T07:06:02.322 INFO:teuthology.orchestra.run.smithi012.stderr:user 0m0.875s 2024-04-27T07:06:02.322 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m0.121s 2024-04-27T07:06:02.322 INFO:tasks.ceph:Archiving logs... 2024-04-27T07:06:02.323 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037/remote/smithi012/log 2024-04-27T07:06:02.323 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T07:06:02.608 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037/remote/smithi053/log 2024-04-27T07:06:02.609 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T07:06:02.707 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-27T07:06:02.724 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-27T07:06:02.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T07:06:02.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T07:06:02.797 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-27T07:06:02.808 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-27T07:06:02.808 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T07:06:02.811 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T07:06:02.822 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2024-04-27T07:06:02.822 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2024-04-27T07:06:02.822 INFO:teuthology.orchestra.run.smithi012.stdout:+hv01.front.sepi 67.205.162.81 3 u 15 64 77 0.087 -0.895 0.137 2024-04-27T07:06:02.822 INFO:teuthology.orchestra.run.smithi012.stdout:*hv02.front.sepi 63.231.80.2 3 u 19 64 77 0.103 +0.130 0.131 2024-04-27T07:06:02.822 INFO:teuthology.orchestra.run.smithi012.stdout:+hv03.front.sepi 74.6.168.72 3 u 21 64 77 0.098 -2.143 0.366 2024-04-27T07:06:02.822 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:06:02.825 INFO:teuthology.orchestra.run.smithi053.stdout: remote refid st t when poll reach delay offset jitter 2024-04-27T07:06:02.826 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================== 2024-04-27T07:06:02.826 INFO:teuthology.orchestra.run.smithi053.stdout:*hv01.front.sepi 67.205.162.81 3 u 16 64 77 0.133 -0.744 1.356 2024-04-27T07:06:02.826 INFO:teuthology.orchestra.run.smithi053.stdout:+hv02.front.sepi 63.231.80.2 3 u 17 64 77 0.088 +0.292 1.366 2024-04-27T07:06:02.826 INFO:teuthology.orchestra.run.smithi053.stdout:+hv03.front.sepi 74.6.168.72 3 u 18 64 77 0.110 -1.943 1.481 2024-04-27T07:06:02.826 INFO:teuthology.orchestra.run.smithi053.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-27T07:06:02.827 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-27T07:06:02.890 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-27T07:06:02.890 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-27T07:06:02.901 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-27T07:06:02.919 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-27T07:06:02.930 INFO:teuthology.task.internal:Duration was 786.890177 seconds 2024-04-27T07:06:02.931 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-27T07:06:02.940 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-27T07:06:02.941 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T07:06:02.943 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T07:06:02.982 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-27T07:06:02.982 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-04-27T07:06:02.982 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T07:06:03.035 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-04-27T07:06:03.035 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T07:06:03.046 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-27T07:06:03.046 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T07:06:03.078 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T07:06:03.112 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-27T07:06:03.113 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T07:06:03.150 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T07:06:03.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-27T07:06:03.224 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-27T07:06:03.225 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T07:06:03.238 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T07:06:03.252 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-27T07:06:03.263 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T07:06:03.282 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T07:06:03.292 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-04-27T07:06:03.304 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-04-27T07:06:03.317 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T07:06:03.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:06:03.353 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T07:06:03.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T07:06:03.362 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-27T07:06:03.373 INFO:teuthology.task.internal:Transferring archived files... 2024-04-27T07:06:03.373 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037/remote/smithi012 2024-04-27T07:06:03.374 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T07:06:03.417 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037/remote/smithi053 2024-04-27T07:06:03.417 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T07:06:03.442 INFO:teuthology.task.internal:Removing archive directory... 2024-04-27T07:06:03.442 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T07:06:03.458 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T07:06:03.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-27T07:06:03.494 INFO:teuthology.task.internal:Not uploading archives. 2024-04-27T07:06:03.495 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-27T07:06:03.508 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-27T07:06:03.508 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T07:06:03.510 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T07:06:03.514 INFO:teuthology.orchestra.run.smithi012.stdout: 658469 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 07:06 /home/ubuntu/cephtest 2024-04-27T07:06:03.534 INFO:teuthology.orchestra.run.smithi053.stdout: 658643 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 07:06 /home/ubuntu/cephtest 2024-04-27T07:06:03.535 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-27T07:06:03.554 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-27T07:06:03.578 INFO:teuthology.nuke:Checking targets against current locks 2024-04-27T07:06:03.605 DEBUG:teuthology.nuke:shortname: smithi012 2024-04-27T07:06:03.605 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T07:06:03.629 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 06:44:27.756996', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBByKoS5wtV294iofbbkD3c3JmT9A/H+k//+OspIbHvtlCiRYHBcAtpsx8ZGlqxT7/kBjYs8ihx6uCGZlvOLDaJU='} 2024-04-27T07:06:03.633 DEBUG:teuthology.nuke:shortname: smithi053 2024-04-27T07:06:03.633 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T07:06:03.658 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_20:24:13-fs-main-distro-default-smithi/7672037', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 06:44:27.755952', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMRr6fEgH9O7yAPsE7IESr/UoTs+hNtQFq2EUtgh9JWnpxRplmFRVRRPDppla624f4eBnmwhreWF1I7Z9teX5fk='} 2024-04-27T07:06:03.686 INFO:teuthology.orchestra.console.smithi012:Power off 2024-04-27T07:06:03.686 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-27T07:06:03.704 INFO:teuthology.orchestra.console.smithi053:Power off 2024-04-27T07:06:03.704 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-27T07:06:03.716 DEBUG:teuthology.orchestra.console.smithi012:power off output: Chassis Power Control: Down/Off 2024-04-27T07:06:03.717 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:06:03.730 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2024-04-27T07:06:03.730 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:06:03.743 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2024-04-27T07:06:03.757 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-04-27T07:06:03.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-04-27T07:06:07.745 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:06:07.758 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:06:07.857 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2024-04-27T07:06:07.869 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2024-04-27T07:06:11.858 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:06:11.869 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-27T07:06:11.970 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2024-04-27T07:06:11.980 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2024-04-27T07:06:12.071 INFO:teuthology.orchestra.console.smithi012:Power off completed 2024-04-27T07:06:12.081 INFO:teuthology.orchestra.console.smithi053:Power off completed 2024-04-27T07:06:12.186 INFO:teuthology.run:Summary data: description: fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} duration: 786.8901770114899 failure_reason: 'Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=669383f5e6d6454e8b01af5d51ab1fd8 status: fail success: false 2024-04-27T07:06:12.186 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T07:06:12.265 INFO:teuthology.run:FAIL